Dash/Plotly - long_callback 失败，后端 celery/redis
Dash/Plotly - long_callback fails with celery/redis backend
总结
我一直在开发一个使用 long_callback 的 dash 应用程序，为了开发，我一直在为我的 long_callback_manager 使用 diskcache 后端，正如指南所推荐的我在这里找到：https://dash.plotly.com/long-callbacks
当我尝试 运行 使用 gunicorn 连接我的应用程序时，它无法启动，因为 diskcache 明显有问题。因此，我决定切换到 celery/redis 后端，因为无论如何都建议将其用于生产。
我有一个 redis 服务器 运行ning（使用 PONG 正确响应 redis-cli ping），然后再次启动该应用程序。这次启动正常，所有正常的回调都有效，但 long_callback 不起作用。
详情：
- 页面或多或少挂起，页面标题在正常标题和
Updating... 标题之间闪烁，表明应用认为它正在“等待”来自 long_callback.
long_callback 的 运行ning 参数设置的值设置为它们的起始值，表明应用程序识别 long_callback 应该 运行 .- 通过将 print 语句作为
long_callback 函数的第一行并看到它没有打印，我确定该函数永远不会启动。
- 使用 gunicorn 和不使用 gunicorn 都会失败。
这些细节都指向 celery/redis 后端的问题。 client/browser 和服务器的 stdout/sterr.
均未显示任何错误
如何让 celery/redis 后端工作？
更新：在意识到正在使用 __name__ 变量并且其值根据引用它的文件而变化后，我还尝试移动创建 celery_app 的代码和 LONG_CALLBACK_MANAGER 到 app.py，无济于事。完全相同的事情发生了。
代码
app.py
import dash
import dash_bootstrap_components as dbc
from website.layout_main import define_callbacks, layout
from website.long_callback_manager import LONG_CALLBACK_MANAGER
app = dash.Dash(
__name__,
update_title="Loading...",
external_stylesheets=[
dbc.themes.BOOTSTRAP,
"https://codepen.io/chriddyp/pen/bWLwgP.css"
],
long_callback_manager=LONG_CALLBACK_MANAGER
)
app.title = "CS 236 | Project Submissions"
app.layout = layout
define_callbacks(app)
server = app.server # expose for gunicorn
if __name__ == "__main__":
app.run_server(debug=True, host="0.0.0.0")
website/long_callback_manager.py with diskcache（功能）
import os
import shutil
import diskcache
from dash.long_callback import DiskcacheLongCallbackManager
from util import RUN_DIR
cache_dir = os.path.join(RUN_DIR, "callback_cache")
shutil.rmtree(cache_dir, ignore_errors=True) # ok if it didn't exist
cache = diskcache.Cache(cache_dir)
LONG_CALLBACK_MANAGER = DiskcacheLongCallbackManager(cache)
website/long_callback_manager.py 与 celery/redis（无效）
from dash.long_callback import CeleryLongCallbackManager
from celery import Celery
celery_app = Celery(
__name__,
broker="redis://localhost:6379/0",
backend="redis://localhost:6379/1"
)
LONG_CALLBACK_MANAGER = CeleryLongCallbackManager(celery_app)
website/layout_main.py
from typing import Union
import dash
import dash_bootstrap_components as dbc
from dash import dcc, html
from dash.dependencies import Input, Output, State
from util.authenticator import authenticate
from website import ID_LOGIN_STORE, NET_ID, PASSWORD
from website.tabs.config import define_config_callbacks, layout as config_layout
from website.tabs.log import define_log_callbacks, layout as log_layout
from website.tabs.submit import define_submit_callbacks, layout as submit_layout
from website.util import AUTH_FAILED_MESSAGE, STYLE_RED
# cache
LOGIN_INFO_EMPTY = {NET_ID: None, PASSWORD: None}
# button display modes
VISIBLE = "inline-block"
HIDDEN = "none"
# header
ID_LOGIN_BUTTON = "login-button"
ID_LOGGED_IN_AS = "logged-in-as"
ID_LOGOUT_BUTTON = "logout-button"
# tabs
ID_TAB_SELECTOR = "tab-selector"
ID_SUBMIT_TAB = "submit-tab"
ID_LOG_TAB = "log-tab"
ID_CONFIG_TAB = "config-tab"
# login modal
ID_LOGIN_MODAL = "login-modal"
ID_LOGIN_MODAL_NET_ID = "login-modal-net-id"
ID_LOGIN_MODAL_PASSWORD = "login-modal-password"
ID_LOGIN_MODAL_MESSAGE = "login-modal-message"
ID_LOGIN_MODAL_CANCEL = "login-modal-cancel"
ID_LOGIN_MODAL_ACCEPT = "login-modal-accept"
# logout modal
ID_LOGOUT_MODAL = "logout-modal"
ID_LOGOUT_MODAL_CANCEL = "logout-modal-cancel"
ID_LOGOUT_MODAL_ACCEPT = "logout-modal-accept"
layout = html.Div([
dcc.Store(id=ID_LOGIN_STORE, storage_type="session", data=LOGIN_INFO_EMPTY),
html.Div(
[
html.H2("BYU CS 236 - Project Submission Website", style={"marginLeft": "10px"}),
html.Div(
[
html.Div(id=ID_LOGGED_IN_AS, style={"display": HIDDEN, "marginRight": "10px"}),
html.Button("Log in", id=ID_LOGIN_BUTTON, style={"display": VISIBLE}),
html.Button("Log out", id=ID_LOGOUT_BUTTON, style={"display": HIDDEN})
],
style={
"marginRight": "25px",
"display": "flex",
"alignItems": "center"
}
)
],
style={
"height": "100px",
"marginLeft": "10px",
"marginRight": "10px",
"display": "flex",
"alignItems": "center",
"justifyContent": "space-between"
}
),
dcc.Tabs(id=ID_TAB_SELECTOR, value=ID_SUBMIT_TAB, children=[
dcc.Tab(submit_layout, label="New Submission", value=ID_SUBMIT_TAB),
dcc.Tab(log_layout, label="Submission Logs", value=ID_LOG_TAB),
dcc.Tab(config_layout, label="View Configuration", value=ID_CONFIG_TAB)
]),
dbc.Modal(
[
dbc.ModalHeader("Log In"),
dbc.ModalBody([
html.Div(
[
html.Label("BYU Net ID:", style={"marginRight": "10px"}),
dcc.Input(
id=ID_LOGIN_MODAL_NET_ID,
type="text",
autoComplete="username",
value="",
style={"marginRight": "30px"}
)
],
style={
"marginBottom": "5px",
"display": "flex",
"alignItems": "center",
"justifyContent": "flex-end"
}
),
html.Div(
[
html.Label("Submission Password:", style={"marginRight": "10px"}),
dcc.Input(
id=ID_LOGIN_MODAL_PASSWORD,
type="password",
autoComplete="current-password",
value="",
style={"marginRight": "30px"}
)
],
style={
"display": "flex",
"alignItems": "center",
"justifyContent": "flex-end"
}
),
html.Div(id=ID_LOGIN_MODAL_MESSAGE, style={"textAlign": "center", "marginTop": "10px"})
]),
dbc.ModalFooter([
html.Button("Cancel", id=ID_LOGIN_MODAL_CANCEL),
html.Button("Log In", id=ID_LOGIN_MODAL_ACCEPT)
])
],
id=ID_LOGIN_MODAL,
is_open=False
),
dbc.Modal(
[
dbc.ModalHeader("Log Out"),
dbc.ModalBody("Are you sure you want to log out?"),
dbc.ModalFooter([
html.Button("Stay Logged In", id=ID_LOGOUT_MODAL_CANCEL),
html.Button("Log Out", id=ID_LOGOUT_MODAL_ACCEPT)
])
],
id=ID_LOGOUT_MODAL,
is_open=False
)
])
def on_click_login_modal_accept(net_id: Union[str, None], password: Union[str, None]) -> Union[str, None]:
# validate
if net_id is None or net_id == "":
return "BYU Net ID is required."
if password is None or password == "":
return "Submission Password is required."
# authenticate
auth_success = authenticate(net_id, password)
if auth_success:
return None
else:
return AUTH_FAILED_MESSAGE
def define_callbacks(app: dash.Dash):
@app.callback(Output(ID_LOGIN_MODAL, "is_open"),
Output(ID_LOGIN_MODAL_MESSAGE, "children"),
Output(ID_LOGOUT_MODAL, "is_open"),
Output(ID_LOGIN_STORE, "data"),
Input(ID_LOGIN_BUTTON, "n_clicks"),
Input(ID_LOGIN_MODAL_CANCEL, "n_clicks"),
Input(ID_LOGIN_MODAL_ACCEPT, "n_clicks"),
Input(ID_LOGOUT_BUTTON, "n_clicks"),
Input(ID_LOGOUT_MODAL_CANCEL, "n_clicks"),
Input(ID_LOGOUT_MODAL_ACCEPT, "n_clicks"),
State(ID_LOGIN_MODAL_NET_ID, "value"),
State(ID_LOGIN_MODAL_PASSWORD, "value"),
prevent_initial_call=True)
def on_login_logout_clicked(
n_login_clicks: int,
n_login_cancel_clicks: int,
n_login_accept_clicks: int,
n_logout_clicks: int,
n_logout_cancel_clicks: int,
n_logout_accept_clicks: int,
net_id: str,
password: str):
ctx = dash.callback_context
btn_id = ctx.triggered[0]["prop_id"].split(".")[0]
if btn_id == ID_LOGIN_BUTTON:
# show the login modal (with no message)
return True, None, dash.no_update, dash.no_update
elif btn_id == ID_LOGIN_MODAL_CANCEL:
# hide the login modal
return False, dash.no_update, dash.no_update, dash.no_update
elif btn_id == ID_LOGIN_MODAL_ACCEPT:
# try to actually log in
error_message = on_click_login_modal_accept(net_id, password)
if error_message is None: # login success!
# hide the modal, update the login store
return False, dash.no_update, dash.no_update, {NET_ID: net_id, PASSWORD: password}
else: # login failed
# show the message and keep the modal open
return dash.no_update, html.Span(error_message, style=STYLE_RED), dash.no_update, dash.no_update
elif btn_id == ID_LOGOUT_BUTTON:
# show the logout modal
return dash.no_update, dash.no_update, True, dash.no_update
elif btn_id == ID_LOGOUT_MODAL_CANCEL:
# hide the logout modal
return dash.no_update, dash.no_update, False, dash.no_update
elif btn_id == ID_LOGOUT_MODAL_ACCEPT:
# hide the logout modal and clear the login store
return dash.no_update, dash.no_update, False, LOGIN_INFO_EMPTY
else: # error
print(f"unknown button id: {btn_id}") # TODO: better logging
return [dash.no_update] * 4 # one for each Output
@app.callback(Output(ID_LOGIN_BUTTON, "style"),
Output(ID_LOGGED_IN_AS, "children"),
Output(ID_LOGGED_IN_AS, "style"),
Output(ID_LOGOUT_BUTTON, "style"),
Input(ID_LOGIN_STORE, "data"),
State(ID_LOGIN_BUTTON, "style"),
State(ID_LOGGED_IN_AS, "style"),
State(ID_LOGOUT_BUTTON, "style"))
def on_login_data_changed(login_store, login_style, logged_in_as_style, logout_style):
# just in case no style is provided
if login_style is None:
login_style = dict()
if logged_in_as_style is None:
logged_in_as_style = dict()
if logout_style is None:
logout_style = dict()
# are they logged in or not?
if login_store[NET_ID] is None or login_store[PASSWORD] is None:
# not logged in
login_style["display"] = VISIBLE
logged_in_as_style["display"] = HIDDEN
logout_style["display"] = HIDDEN
return login_style, None, logged_in_as_style, logout_style
else: # yes logged in
login_style["display"] = HIDDEN
logged_in_as_style["display"] = VISIBLE
logout_style["display"] = VISIBLE
return login_style, f"Logged in as '{login_store[NET_ID]}'", logged_in_as_style, logout_style
# define callbacks for all of the tabs
define_submit_callbacks(app)
define_log_callbacks(app)
define_config_callbacks(app)
website/tabs/submit.py
import os
import time
from io import StringIO
from typing import Callable, Dict, Union
import dash
import dash_bootstrap_components as dbc
import dash_gif_component as gif
from dash import dcc, html
from dash.dependencies import Input, Output, State
from dash.exceptions import PreventUpdate
from config.loaded_config import CONFIG
from driver.passoff_driver import PassoffDriver
from util.authenticator import authenticate
from website import ID_LOGIN_STORE, NET_ID, PASSWORD
from website.util import AUTH_FAILED_MESSAGE, save_to_submit, STYLE_DIV_VISIBLE, STYLE_DIV_VISIBLE_TOP_MARGIN, STYLE_HIDDEN, text_html_colorizer
# submit tab IDs
ID_SUBMISSION_ROOT_DIV = "submission-root-div"
ID_SUBMIT_PROJECT_NUMBER_RADIO = "submit-project-number-radio"
ID_UPLOAD_BUTTON = "upload-button"
ID_UPLOAD_CONTENTS = "upload-contents"
ID_FILE_NAME_DISPLAY = "file-name-display"
ID_SUBMISSION_SUBMIT_BUTTON = "submission-submit-button"
ID_SUBMISSION_OUTPUT = "submission-output"
ID_SUBMISSION_LOADING = "submission-loading"
# clear/refresh to submit again
ID_SUBMISSION_REFRESH_BUTTON = "submission-refresh-button"
ID_SUBMISSION_REFRESH_DIV = "submission-refresh-div"
ID_SUBMISSION_RESETTING_STORE = "submission-resetting-store"
# info modal
ID_SUBMISSION_INFO_MODAL = "submission-info-modal"
ID_SUBMISSION_INFO_MODAL_MESSAGE = "submission-info-modal-message"
ID_SUBMISSION_INFO_MODAL_ACCEPT = "submission-info-modal-accept"
# submission confirmation modal
ID_SUBMISSION_CONFIRMATION_MODAL = "submission-confirmation-modal"
ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL = "submission-confirmation-modal-cancel"
ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT = "submission-confirmation-modal-accept"
# store to trigger submission
ID_SUBMISSION_TRIGGER_STORE = "submission-trigger-store"
LAYOUT_DEFAULT_CONTENTS = [
html.H3("Upload New Submission"),
html.P("Which project are you submitting?"),
dcc.RadioItems(
id=ID_SUBMIT_PROJECT_NUMBER_RADIO,
options=[{
"label": f" Project {proj_num}",
"value": proj_num
} for proj_num in range(1, CONFIG.n_projects + 1)]
),
html.Br(),
html.P("Upload your .zip file here:"),
html.Div(
[
dcc.Upload(
html.Button("Select File", id=ID_UPLOAD_BUTTON),
id=ID_UPLOAD_CONTENTS,
multiple=False
),
html.Pre("No File Selected", id=ID_FILE_NAME_DISPLAY, style={"marginLeft": "10px"})
],
style={
"display": "flex",
"justifyContent": "flex-start",
"alignItems": "center"
}
),
html.Button("Submit", id=ID_SUBMISSION_SUBMIT_BUTTON, style={"marginTop": "20px"}),
html.Div(id=ID_SUBMISSION_OUTPUT, style=STYLE_HIDDEN),
html.Div(
html.Div(
gif.GifPlayer(
gif=os.path.join("assets", "loading.gif"),
still=os.path.join("assets", "loading.png"),
alt="loading symbol",
autoplay=True
),
style={"zoom": "0.2"}
),
id=ID_SUBMISSION_LOADING,
style=STYLE_HIDDEN
),
html.Div(
[
html.P("Reset the page to submit again:"),
html.Button("Reset", id=ID_SUBMISSION_REFRESH_BUTTON),
],
id=ID_SUBMISSION_REFRESH_DIV,
style=STYLE_HIDDEN
),
dbc.Modal(
[
dbc.ModalHeader("Try Again"),
dbc.ModalBody(id=ID_SUBMISSION_INFO_MODAL_MESSAGE),
dbc.ModalFooter([
html.Button("OK", id=ID_SUBMISSION_INFO_MODAL_ACCEPT)
])
],
id=ID_SUBMISSION_INFO_MODAL,
is_open=False
),
dbc.Modal(
[
dbc.ModalHeader("Confirm Submission"),
dbc.ModalBody("Are you sure you want to officially submit?"),
dbc.ModalFooter([
html.Button("Cancel", id=ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL),
html.Button("Submit", id=ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT)
])
],
id=ID_SUBMISSION_CONFIRMATION_MODAL,
is_open=False
)
]
layout = html.Div(
[
html.Div(LAYOUT_DEFAULT_CONTENTS, id=ID_SUBMISSION_ROOT_DIV),
# having this store outside of the layout that gets reset means the long callback is not triggered
dcc.Store(id=ID_SUBMISSION_TRIGGER_STORE, storage_type="memory", data=False) # data value just flips to trigger the long callback
],
style={
"margin": "10px",
"padding": "10px",
"borderStyle": "double"
}
)
def on_submit_button_clicked(
proj_number: Union[int, None],
file_name: Union[str, None],
file_contents: Union[str, None],
login_store: Union[Dict[str, str], None]) -> Union[str, None]:
# validate
if login_store is None or NET_ID not in login_store or PASSWORD not in login_store:
return "There was a problem with the login store!"
net_id = login_store[NET_ID]
password = login_store[PASSWORD]
if net_id is None or net_id == "" or password is None or password == "":
return "You must log in before submitting."
if proj_number is None:
return "The project number must be selected."
if not (1 <= proj_number <= CONFIG.n_projects):
return "Invalid project selected."
if file_name is None or file_name == "" or file_contents is None or file_contents == "":
return "A zip file must be uploaded to submit."
if not file_name.endswith(".zip"):
return "The uploaded file must be a .zip file."
# all good, it seems; return no error message
return None
def run_submission(proj_number: int, file_contents: str, login_store: Dict[str, str], set_progress: Callable):
# authenticate
print("authenticate")
net_id = login_store[NET_ID]
password = login_store[PASSWORD]
auth_success = authenticate(net_id, password)
if not auth_success:
set_progress([AUTH_FAILED_MESSAGE])
return
# write their zip file to the submit directory
print("save_to_submit")
save_to_submit(proj_number, net_id, file_contents)
# actually submit
print("actually submit")
this_stdout = StringIO()
this_stderr = StringIO()
driver = PassoffDriver(net_id, proj_number, use_user_input=False, stdout=this_stdout, stderr=this_stderr)
driver.start() # runs in a thread of this same process
while True: # make sure we print the output at least once, even it it finishes super fast
time.sleep(1) # check output regularly
# TODO: change saved final_result in PassoffDriver to diff/error info?
# show results to the user
output = list()
output.append(html.P(f"submission for Net ID '{net_id}', project {proj_number}"))
stdout_val = this_stdout.getvalue()
output.append(html.Pre(text_html_colorizer(stdout_val)))
# output.append(html.Br())
stderr_val = this_stderr.getvalue()
if stderr_val != "":
output.append(html.P("ERRORS:"))
output.append(html.Pre(text_html_colorizer(stderr_val)))
set_progress([output])
if not driver.is_alive(): # once it finishes, we're done too
print("driver finished")
# TODO: tack on diff info?
if CONFIG.expose_test_cases and driver.final_result is not None and driver.final_result.has_failure_details:
output.append(html.P(html.B("Use the \"Submission Logs\" tab to get more detailed information."), style={"marginTop": "20px"}))
set_progress([output])
break
def define_submit_callbacks(app: dash.Dash):
@app.callback(Output(ID_FILE_NAME_DISPLAY, "children"),
Input(ID_UPLOAD_CONTENTS, "filename"),
prevent_initial_call=True)
def on_select_file(filename: str):
if filename is None or filename == "":
return "No File Selected"
return filename
@app.callback(Output(ID_SUBMISSION_CONFIRMATION_MODAL, "is_open"),
Output(ID_SUBMISSION_INFO_MODAL, "is_open"),
Output(ID_SUBMISSION_INFO_MODAL_MESSAGE, "children"),
Output(ID_SUBMISSION_TRIGGER_STORE, "data"),
Input(ID_SUBMISSION_SUBMIT_BUTTON, "n_clicks"),
Input(ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL, "n_clicks"),
Input(ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT, "n_clicks"),
Input(ID_SUBMISSION_INFO_MODAL_ACCEPT, "n_clicks"),
State(ID_SUBMIT_PROJECT_NUMBER_RADIO, "value"),
State(ID_UPLOAD_CONTENTS, "filename"),
State(ID_UPLOAD_CONTENTS, "contents"),
State(ID_LOGIN_STORE, "data"),
State(ID_SUBMISSION_TRIGGER_STORE, "data"),
prevent_initial_call=True)
def on_submission_submit_clicked(
n_submit_clicks: int,
n_confirmation_cancel_clicks: int,
n_confirmation_accept_clicks: int,
n_info_accept_clicks: int,
proj_number: int,
file_name: str,
file_contents: str,
login_store: Dict[str, Union[str, None]],
submission_trigger_store: bool):
ctx = dash.callback_context
trigger_id = ctx.triggered[0]["prop_id"].split(".")[0]
if trigger_id == ID_SUBMISSION_SUBMIT_BUTTON:
if n_submit_clicks is None:
raise PreventUpdate
# validate
error_message = on_submit_button_clicked(proj_number, file_name, file_contents, login_store)
if error_message is None: # good to go
# show the confirmation modal
return True, dash.no_update, dash.no_update, dash.no_update
else:
# show the error message in the info modal
return dash.no_update, True, error_message, dash.no_update
elif trigger_id == ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL:
if n_confirmation_cancel_clicks is None:
raise PreventUpdate
# hide the confirmation modal
return False, dash.no_update, dash.no_update, dash.no_update
elif trigger_id == ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT:
if n_confirmation_accept_clicks is None:
raise PreventUpdate
# hide the confirmation modal and trigger on_submit_confirmed
return False, dash.no_update, dash.no_update, not submission_trigger_store # just flip the value, whatever it is
elif trigger_id == ID_SUBMISSION_INFO_MODAL_ACCEPT:
if n_info_accept_clicks is None:
raise PreventUpdate
# hide the info modal
return dash.no_update, False, dash.no_update, dash.no_update
else: # error
print(f"unknown button id: {trigger_id}") # TODO: better logging
return [dash.no_update] * 4 # one for each Output
@app.long_callback(
progress=[Output(ID_SUBMISSION_OUTPUT, "children")],
progress_default=[dash.no_update], # I'll set stuff manually, thank you very much
output=[Output(ID_SUBMISSION_REFRESH_DIV, "style")],
inputs=[
Input(ID_SUBMISSION_TRIGGER_STORE, "data"),
State(ID_SUBMIT_PROJECT_NUMBER_RADIO, "value"),
State(ID_UPLOAD_CONTENTS, "contents"),
State(ID_LOGIN_STORE, "data")
],
running=[ # hide the submit button when it starts running, then keep it hidden so they have to refresh to submit again
(Output(ID_SUBMISSION_SUBMIT_BUTTON, "style"), STYLE_HIDDEN, STYLE_HIDDEN),
(Output(ID_SUBMISSION_LOADING, "style"), STYLE_DIV_VISIBLE, STYLE_HIDDEN),
(Output(ID_SUBMISSION_OUTPUT, "style"), STYLE_DIV_VISIBLE_TOP_MARGIN, STYLE_DIV_VISIBLE_TOP_MARGIN)
],
prevent_initial_call=True)
def on_submit_confirmed(
set_progress: Callable,
submission_trigger_store: bool,
proj_number: int,
file_contents,
login_store):
print("start of long callback")
set_progress([None])
print("after update progress to None")
# actually run the submission
run_submission(proj_number, file_contents, login_store, set_progress)
print("finished run_submission")
# wait a sec to make sure all `set_progress` calls can finish
time.sleep(1)
# show the "clear page" message/button
print("about to return")
return [STYLE_DIV_VISIBLE_TOP_MARGIN]
@app.callback(Output(ID_SUBMISSION_ROOT_DIV, "children"),
Input(ID_SUBMISSION_REFRESH_BUTTON, "n_clicks"),
prevent_initial_call=True)
def on_submission_refresh_clicked(n_refresh_clicks: int):
# reset everything
return LAYOUT_DEFAULT_CONTENTS
环境
python版本
$ python --version
Python 3.9.6
已安装的软件包
$ pip list
Package Version
------------------------- ---------
amqp 5.0.6
billiard 3.6.4.0
Brotli 1.0.9
celery 5.1.2
click 7.1.2
click-didyoumean 0.0.3
click-plugins 1.1.1
click-repl 0.2.0
dash 2.0.0
dash-bootstrap-components 0.13.1
dash-core-components 2.0.0
dash-gif-component 1.1.0
dash-html-components 2.0.0
dash-table 5.0.0
dill 0.3.4
diskcache 5.2.1
Flask 2.0.1
Flask-Compress 1.10.1
greenlet 1.1.1
gunicorn 20.1.0
itsdangerous 2.0.1
Jinja2 3.0.1
kombu 5.1.0
MarkupSafe 2.0.1
multiprocess 0.70.12.2
orjson 3.6.3
pip 21.2.4
plotly 5.3.1
prompt-toolkit 3.0.20
psutil 5.8.0
pytz 2021.1
redis 3.5.3
setuptools 56.0.0
six 1.16.0
SQLAlchemy 1.4.23
tenacity 8.0.1
vine 5.0.0
wcwidth 0.2.5
Werkzeug 2.0.1
wheel 0.37.0
redis
$ redis-server --version
Redis server v=6.2.5 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=2a367e4b809d24de
$ redis-cli ping
PONG
$ curl localhost:6379
curl: (52) Empty reply from server
$ sudo ss -lptn 'sport = :6379'
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 511 127.0.0.1:6379 0.0.0.0:* users:(("redis-server",pid=373,fd=6))
OS 详情
Windows、WSL 2、Ubuntu
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
从 plotly 社区论坛重新发布解决方案：
https://community.plotly.com/t/long-callback-with-celery-redis-how-to-get-the-example-app-work/57663
总结
为了使长回调起作用，我需要启动 3 个协同工作的独立进程：
- Redis 服务器：
redis-server
- Celery 应用程序：
celery -A app.celery worker --loglevel=INFO
- Dash 应用程序：
python app.py
上面列出的命令是最简单的版本。使用的完整命令在适当修改后进一步给出。
详情
我将 celery 应用程序的声明从 src/website/long_callback_manager.py 移到了 src/app.py 以便于外部访问：
import dash
import dash_bootstrap_components as dbc
from celery import Celery
from dash.long_callback import CeleryLongCallbackManager
from website.layout_main import define_callbacks, layout
celery_app = Celery(
__name__,
broker="redis://localhost:6379/0",
backend="redis://localhost:6379/1"
)
LONG_CALLBACK_MANAGER = CeleryLongCallbackManager(celery_app)
app = dash.Dash(
__name__,
update_title="Loading...",
external_stylesheets=[
dbc.themes.BOOTSTRAP,
"https://codepen.io/chriddyp/pen/bWLwgP.css"
],
long_callback_manager=LONG_CALLBACK_MANAGER
)
app.title = "CS 236 | Project Submissions"
app.layout = layout
define_callbacks(app)
server = app.server # expose for gunicorn
if __name__ == "__main__":
app.run_server(debug=True, host="0.0.0.0")
然后我使用以下 bash 脚本来简化启动一切的过程：
#!/bin/bash
set -e # quit on any error
# make sure the redis server is running
if ! redis-cli ping > /dev/null 2>&1; then
redis-server --daemonize yes --bind 127.0.0.1
redis-cli ping > /dev/null 2>&1 # the script halts if redis is not now running (failed to start)
fi
# activate the venv that has our things installed with pip
. venv/bin/activate
# make sure it can find the python modules, but still run from this directory
export PYTHONPATH=src
# make sure we have a log directory
mkdir -p Log
# start the celery thing
celery -A app.celery_app worker --loglevel=INFO >> Log/celery_info.log 2>&1 &
# start the server
gunicorn --workers=4 --name=passoff_website_server --bind=127.0.0.1:8050 app:server >> Log/gunicorn.log 2>&1
此脚本的进程然后是 celery 和 gunicorn 子进程的父进程，并且所有这些都可以通过终止父进程作为一个包终止。
总结
我一直在开发一个使用 long_callback 的 dash 应用程序，为了开发，我一直在为我的 long_callback_manager 使用 diskcache 后端，正如指南所推荐的我在这里找到：https://dash.plotly.com/long-callbacks
当我尝试 运行 使用 gunicorn 连接我的应用程序时，它无法启动，因为 diskcache 明显有问题。因此，我决定切换到 celery/redis 后端，因为无论如何都建议将其用于生产。
我有一个 redis 服务器 运行ning（使用 PONG 正确响应 redis-cli ping），然后再次启动该应用程序。这次启动正常，所有正常的回调都有效，但 long_callback 不起作用。
详情：
- 页面或多或少挂起，页面标题在正常标题和
Updating...标题之间闪烁，表明应用认为它正在“等待”来自long_callback. long_callback的 运行ning 参数设置的值设置为它们的起始值，表明应用程序识别long_callback应该 运行 .- 通过将 print 语句作为
long_callback函数的第一行并看到它没有打印，我确定该函数永远不会启动。 - 使用 gunicorn 和不使用 gunicorn 都会失败。
这些细节都指向 celery/redis 后端的问题。 client/browser 和服务器的 stdout/sterr.
均未显示任何错误如何让 celery/redis 后端工作？
更新：在意识到正在使用 __name__ 变量并且其值根据引用它的文件而变化后，我还尝试移动创建 celery_app 的代码和 LONG_CALLBACK_MANAGER 到 app.py，无济于事。完全相同的事情发生了。
代码
app.py
import dash
import dash_bootstrap_components as dbc
from website.layout_main import define_callbacks, layout
from website.long_callback_manager import LONG_CALLBACK_MANAGER
app = dash.Dash(
__name__,
update_title="Loading...",
external_stylesheets=[
dbc.themes.BOOTSTRAP,
"https://codepen.io/chriddyp/pen/bWLwgP.css"
],
long_callback_manager=LONG_CALLBACK_MANAGER
)
app.title = "CS 236 | Project Submissions"
app.layout = layout
define_callbacks(app)
server = app.server # expose for gunicorn
if __name__ == "__main__":
app.run_server(debug=True, host="0.0.0.0")
website/long_callback_manager.py with diskcache（功能）
import os
import shutil
import diskcache
from dash.long_callback import DiskcacheLongCallbackManager
from util import RUN_DIR
cache_dir = os.path.join(RUN_DIR, "callback_cache")
shutil.rmtree(cache_dir, ignore_errors=True) # ok if it didn't exist
cache = diskcache.Cache(cache_dir)
LONG_CALLBACK_MANAGER = DiskcacheLongCallbackManager(cache)
website/long_callback_manager.py 与 celery/redis（无效）
from dash.long_callback import CeleryLongCallbackManager
from celery import Celery
celery_app = Celery(
__name__,
broker="redis://localhost:6379/0",
backend="redis://localhost:6379/1"
)
LONG_CALLBACK_MANAGER = CeleryLongCallbackManager(celery_app)
website/layout_main.py
from typing import Union
import dash
import dash_bootstrap_components as dbc
from dash import dcc, html
from dash.dependencies import Input, Output, State
from util.authenticator import authenticate
from website import ID_LOGIN_STORE, NET_ID, PASSWORD
from website.tabs.config import define_config_callbacks, layout as config_layout
from website.tabs.log import define_log_callbacks, layout as log_layout
from website.tabs.submit import define_submit_callbacks, layout as submit_layout
from website.util import AUTH_FAILED_MESSAGE, STYLE_RED
# cache
LOGIN_INFO_EMPTY = {NET_ID: None, PASSWORD: None}
# button display modes
VISIBLE = "inline-block"
HIDDEN = "none"
# header
ID_LOGIN_BUTTON = "login-button"
ID_LOGGED_IN_AS = "logged-in-as"
ID_LOGOUT_BUTTON = "logout-button"
# tabs
ID_TAB_SELECTOR = "tab-selector"
ID_SUBMIT_TAB = "submit-tab"
ID_LOG_TAB = "log-tab"
ID_CONFIG_TAB = "config-tab"
# login modal
ID_LOGIN_MODAL = "login-modal"
ID_LOGIN_MODAL_NET_ID = "login-modal-net-id"
ID_LOGIN_MODAL_PASSWORD = "login-modal-password"
ID_LOGIN_MODAL_MESSAGE = "login-modal-message"
ID_LOGIN_MODAL_CANCEL = "login-modal-cancel"
ID_LOGIN_MODAL_ACCEPT = "login-modal-accept"
# logout modal
ID_LOGOUT_MODAL = "logout-modal"
ID_LOGOUT_MODAL_CANCEL = "logout-modal-cancel"
ID_LOGOUT_MODAL_ACCEPT = "logout-modal-accept"
layout = html.Div([
dcc.Store(id=ID_LOGIN_STORE, storage_type="session", data=LOGIN_INFO_EMPTY),
html.Div(
[
html.H2("BYU CS 236 - Project Submission Website", style={"marginLeft": "10px"}),
html.Div(
[
html.Div(id=ID_LOGGED_IN_AS, style={"display": HIDDEN, "marginRight": "10px"}),
html.Button("Log in", id=ID_LOGIN_BUTTON, style={"display": VISIBLE}),
html.Button("Log out", id=ID_LOGOUT_BUTTON, style={"display": HIDDEN})
],
style={
"marginRight": "25px",
"display": "flex",
"alignItems": "center"
}
)
],
style={
"height": "100px",
"marginLeft": "10px",
"marginRight": "10px",
"display": "flex",
"alignItems": "center",
"justifyContent": "space-between"
}
),
dcc.Tabs(id=ID_TAB_SELECTOR, value=ID_SUBMIT_TAB, children=[
dcc.Tab(submit_layout, label="New Submission", value=ID_SUBMIT_TAB),
dcc.Tab(log_layout, label="Submission Logs", value=ID_LOG_TAB),
dcc.Tab(config_layout, label="View Configuration", value=ID_CONFIG_TAB)
]),
dbc.Modal(
[
dbc.ModalHeader("Log In"),
dbc.ModalBody([
html.Div(
[
html.Label("BYU Net ID:", style={"marginRight": "10px"}),
dcc.Input(
id=ID_LOGIN_MODAL_NET_ID,
type="text",
autoComplete="username",
value="",
style={"marginRight": "30px"}
)
],
style={
"marginBottom": "5px",
"display": "flex",
"alignItems": "center",
"justifyContent": "flex-end"
}
),
html.Div(
[
html.Label("Submission Password:", style={"marginRight": "10px"}),
dcc.Input(
id=ID_LOGIN_MODAL_PASSWORD,
type="password",
autoComplete="current-password",
value="",
style={"marginRight": "30px"}
)
],
style={
"display": "flex",
"alignItems": "center",
"justifyContent": "flex-end"
}
),
html.Div(id=ID_LOGIN_MODAL_MESSAGE, style={"textAlign": "center", "marginTop": "10px"})
]),
dbc.ModalFooter([
html.Button("Cancel", id=ID_LOGIN_MODAL_CANCEL),
html.Button("Log In", id=ID_LOGIN_MODAL_ACCEPT)
])
],
id=ID_LOGIN_MODAL,
is_open=False
),
dbc.Modal(
[
dbc.ModalHeader("Log Out"),
dbc.ModalBody("Are you sure you want to log out?"),
dbc.ModalFooter([
html.Button("Stay Logged In", id=ID_LOGOUT_MODAL_CANCEL),
html.Button("Log Out", id=ID_LOGOUT_MODAL_ACCEPT)
])
],
id=ID_LOGOUT_MODAL,
is_open=False
)
])
def on_click_login_modal_accept(net_id: Union[str, None], password: Union[str, None]) -> Union[str, None]:
# validate
if net_id is None or net_id == "":
return "BYU Net ID is required."
if password is None or password == "":
return "Submission Password is required."
# authenticate
auth_success = authenticate(net_id, password)
if auth_success:
return None
else:
return AUTH_FAILED_MESSAGE
def define_callbacks(app: dash.Dash):
@app.callback(Output(ID_LOGIN_MODAL, "is_open"),
Output(ID_LOGIN_MODAL_MESSAGE, "children"),
Output(ID_LOGOUT_MODAL, "is_open"),
Output(ID_LOGIN_STORE, "data"),
Input(ID_LOGIN_BUTTON, "n_clicks"),
Input(ID_LOGIN_MODAL_CANCEL, "n_clicks"),
Input(ID_LOGIN_MODAL_ACCEPT, "n_clicks"),
Input(ID_LOGOUT_BUTTON, "n_clicks"),
Input(ID_LOGOUT_MODAL_CANCEL, "n_clicks"),
Input(ID_LOGOUT_MODAL_ACCEPT, "n_clicks"),
State(ID_LOGIN_MODAL_NET_ID, "value"),
State(ID_LOGIN_MODAL_PASSWORD, "value"),
prevent_initial_call=True)
def on_login_logout_clicked(
n_login_clicks: int,
n_login_cancel_clicks: int,
n_login_accept_clicks: int,
n_logout_clicks: int,
n_logout_cancel_clicks: int,
n_logout_accept_clicks: int,
net_id: str,
password: str):
ctx = dash.callback_context
btn_id = ctx.triggered[0]["prop_id"].split(".")[0]
if btn_id == ID_LOGIN_BUTTON:
# show the login modal (with no message)
return True, None, dash.no_update, dash.no_update
elif btn_id == ID_LOGIN_MODAL_CANCEL:
# hide the login modal
return False, dash.no_update, dash.no_update, dash.no_update
elif btn_id == ID_LOGIN_MODAL_ACCEPT:
# try to actually log in
error_message = on_click_login_modal_accept(net_id, password)
if error_message is None: # login success!
# hide the modal, update the login store
return False, dash.no_update, dash.no_update, {NET_ID: net_id, PASSWORD: password}
else: # login failed
# show the message and keep the modal open
return dash.no_update, html.Span(error_message, style=STYLE_RED), dash.no_update, dash.no_update
elif btn_id == ID_LOGOUT_BUTTON:
# show the logout modal
return dash.no_update, dash.no_update, True, dash.no_update
elif btn_id == ID_LOGOUT_MODAL_CANCEL:
# hide the logout modal
return dash.no_update, dash.no_update, False, dash.no_update
elif btn_id == ID_LOGOUT_MODAL_ACCEPT:
# hide the logout modal and clear the login store
return dash.no_update, dash.no_update, False, LOGIN_INFO_EMPTY
else: # error
print(f"unknown button id: {btn_id}") # TODO: better logging
return [dash.no_update] * 4 # one for each Output
@app.callback(Output(ID_LOGIN_BUTTON, "style"),
Output(ID_LOGGED_IN_AS, "children"),
Output(ID_LOGGED_IN_AS, "style"),
Output(ID_LOGOUT_BUTTON, "style"),
Input(ID_LOGIN_STORE, "data"),
State(ID_LOGIN_BUTTON, "style"),
State(ID_LOGGED_IN_AS, "style"),
State(ID_LOGOUT_BUTTON, "style"))
def on_login_data_changed(login_store, login_style, logged_in_as_style, logout_style):
# just in case no style is provided
if login_style is None:
login_style = dict()
if logged_in_as_style is None:
logged_in_as_style = dict()
if logout_style is None:
logout_style = dict()
# are they logged in or not?
if login_store[NET_ID] is None or login_store[PASSWORD] is None:
# not logged in
login_style["display"] = VISIBLE
logged_in_as_style["display"] = HIDDEN
logout_style["display"] = HIDDEN
return login_style, None, logged_in_as_style, logout_style
else: # yes logged in
login_style["display"] = HIDDEN
logged_in_as_style["display"] = VISIBLE
logout_style["display"] = VISIBLE
return login_style, f"Logged in as '{login_store[NET_ID]}'", logged_in_as_style, logout_style
# define callbacks for all of the tabs
define_submit_callbacks(app)
define_log_callbacks(app)
define_config_callbacks(app)
website/tabs/submit.py
import os
import time
from io import StringIO
from typing import Callable, Dict, Union
import dash
import dash_bootstrap_components as dbc
import dash_gif_component as gif
from dash import dcc, html
from dash.dependencies import Input, Output, State
from dash.exceptions import PreventUpdate
from config.loaded_config import CONFIG
from driver.passoff_driver import PassoffDriver
from util.authenticator import authenticate
from website import ID_LOGIN_STORE, NET_ID, PASSWORD
from website.util import AUTH_FAILED_MESSAGE, save_to_submit, STYLE_DIV_VISIBLE, STYLE_DIV_VISIBLE_TOP_MARGIN, STYLE_HIDDEN, text_html_colorizer
# submit tab IDs
ID_SUBMISSION_ROOT_DIV = "submission-root-div"
ID_SUBMIT_PROJECT_NUMBER_RADIO = "submit-project-number-radio"
ID_UPLOAD_BUTTON = "upload-button"
ID_UPLOAD_CONTENTS = "upload-contents"
ID_FILE_NAME_DISPLAY = "file-name-display"
ID_SUBMISSION_SUBMIT_BUTTON = "submission-submit-button"
ID_SUBMISSION_OUTPUT = "submission-output"
ID_SUBMISSION_LOADING = "submission-loading"
# clear/refresh to submit again
ID_SUBMISSION_REFRESH_BUTTON = "submission-refresh-button"
ID_SUBMISSION_REFRESH_DIV = "submission-refresh-div"
ID_SUBMISSION_RESETTING_STORE = "submission-resetting-store"
# info modal
ID_SUBMISSION_INFO_MODAL = "submission-info-modal"
ID_SUBMISSION_INFO_MODAL_MESSAGE = "submission-info-modal-message"
ID_SUBMISSION_INFO_MODAL_ACCEPT = "submission-info-modal-accept"
# submission confirmation modal
ID_SUBMISSION_CONFIRMATION_MODAL = "submission-confirmation-modal"
ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL = "submission-confirmation-modal-cancel"
ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT = "submission-confirmation-modal-accept"
# store to trigger submission
ID_SUBMISSION_TRIGGER_STORE = "submission-trigger-store"
LAYOUT_DEFAULT_CONTENTS = [
html.H3("Upload New Submission"),
html.P("Which project are you submitting?"),
dcc.RadioItems(
id=ID_SUBMIT_PROJECT_NUMBER_RADIO,
options=[{
"label": f" Project {proj_num}",
"value": proj_num
} for proj_num in range(1, CONFIG.n_projects + 1)]
),
html.Br(),
html.P("Upload your .zip file here:"),
html.Div(
[
dcc.Upload(
html.Button("Select File", id=ID_UPLOAD_BUTTON),
id=ID_UPLOAD_CONTENTS,
multiple=False
),
html.Pre("No File Selected", id=ID_FILE_NAME_DISPLAY, style={"marginLeft": "10px"})
],
style={
"display": "flex",
"justifyContent": "flex-start",
"alignItems": "center"
}
),
html.Button("Submit", id=ID_SUBMISSION_SUBMIT_BUTTON, style={"marginTop": "20px"}),
html.Div(id=ID_SUBMISSION_OUTPUT, style=STYLE_HIDDEN),
html.Div(
html.Div(
gif.GifPlayer(
gif=os.path.join("assets", "loading.gif"),
still=os.path.join("assets", "loading.png"),
alt="loading symbol",
autoplay=True
),
style={"zoom": "0.2"}
),
id=ID_SUBMISSION_LOADING,
style=STYLE_HIDDEN
),
html.Div(
[
html.P("Reset the page to submit again:"),
html.Button("Reset", id=ID_SUBMISSION_REFRESH_BUTTON),
],
id=ID_SUBMISSION_REFRESH_DIV,
style=STYLE_HIDDEN
),
dbc.Modal(
[
dbc.ModalHeader("Try Again"),
dbc.ModalBody(id=ID_SUBMISSION_INFO_MODAL_MESSAGE),
dbc.ModalFooter([
html.Button("OK", id=ID_SUBMISSION_INFO_MODAL_ACCEPT)
])
],
id=ID_SUBMISSION_INFO_MODAL,
is_open=False
),
dbc.Modal(
[
dbc.ModalHeader("Confirm Submission"),
dbc.ModalBody("Are you sure you want to officially submit?"),
dbc.ModalFooter([
html.Button("Cancel", id=ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL),
html.Button("Submit", id=ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT)
])
],
id=ID_SUBMISSION_CONFIRMATION_MODAL,
is_open=False
)
]
layout = html.Div(
[
html.Div(LAYOUT_DEFAULT_CONTENTS, id=ID_SUBMISSION_ROOT_DIV),
# having this store outside of the layout that gets reset means the long callback is not triggered
dcc.Store(id=ID_SUBMISSION_TRIGGER_STORE, storage_type="memory", data=False) # data value just flips to trigger the long callback
],
style={
"margin": "10px",
"padding": "10px",
"borderStyle": "double"
}
)
def on_submit_button_clicked(
proj_number: Union[int, None],
file_name: Union[str, None],
file_contents: Union[str, None],
login_store: Union[Dict[str, str], None]) -> Union[str, None]:
# validate
if login_store is None or NET_ID not in login_store or PASSWORD not in login_store:
return "There was a problem with the login store!"
net_id = login_store[NET_ID]
password = login_store[PASSWORD]
if net_id is None or net_id == "" or password is None or password == "":
return "You must log in before submitting."
if proj_number is None:
return "The project number must be selected."
if not (1 <= proj_number <= CONFIG.n_projects):
return "Invalid project selected."
if file_name is None or file_name == "" or file_contents is None or file_contents == "":
return "A zip file must be uploaded to submit."
if not file_name.endswith(".zip"):
return "The uploaded file must be a .zip file."
# all good, it seems; return no error message
return None
def run_submission(proj_number: int, file_contents: str, login_store: Dict[str, str], set_progress: Callable):
# authenticate
print("authenticate")
net_id = login_store[NET_ID]
password = login_store[PASSWORD]
auth_success = authenticate(net_id, password)
if not auth_success:
set_progress([AUTH_FAILED_MESSAGE])
return
# write their zip file to the submit directory
print("save_to_submit")
save_to_submit(proj_number, net_id, file_contents)
# actually submit
print("actually submit")
this_stdout = StringIO()
this_stderr = StringIO()
driver = PassoffDriver(net_id, proj_number, use_user_input=False, stdout=this_stdout, stderr=this_stderr)
driver.start() # runs in a thread of this same process
while True: # make sure we print the output at least once, even it it finishes super fast
time.sleep(1) # check output regularly
# TODO: change saved final_result in PassoffDriver to diff/error info?
# show results to the user
output = list()
output.append(html.P(f"submission for Net ID '{net_id}', project {proj_number}"))
stdout_val = this_stdout.getvalue()
output.append(html.Pre(text_html_colorizer(stdout_val)))
# output.append(html.Br())
stderr_val = this_stderr.getvalue()
if stderr_val != "":
output.append(html.P("ERRORS:"))
output.append(html.Pre(text_html_colorizer(stderr_val)))
set_progress([output])
if not driver.is_alive(): # once it finishes, we're done too
print("driver finished")
# TODO: tack on diff info?
if CONFIG.expose_test_cases and driver.final_result is not None and driver.final_result.has_failure_details:
output.append(html.P(html.B("Use the \"Submission Logs\" tab to get more detailed information."), style={"marginTop": "20px"}))
set_progress([output])
break
def define_submit_callbacks(app: dash.Dash):
@app.callback(Output(ID_FILE_NAME_DISPLAY, "children"),
Input(ID_UPLOAD_CONTENTS, "filename"),
prevent_initial_call=True)
def on_select_file(filename: str):
if filename is None or filename == "":
return "No File Selected"
return filename
@app.callback(Output(ID_SUBMISSION_CONFIRMATION_MODAL, "is_open"),
Output(ID_SUBMISSION_INFO_MODAL, "is_open"),
Output(ID_SUBMISSION_INFO_MODAL_MESSAGE, "children"),
Output(ID_SUBMISSION_TRIGGER_STORE, "data"),
Input(ID_SUBMISSION_SUBMIT_BUTTON, "n_clicks"),
Input(ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL, "n_clicks"),
Input(ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT, "n_clicks"),
Input(ID_SUBMISSION_INFO_MODAL_ACCEPT, "n_clicks"),
State(ID_SUBMIT_PROJECT_NUMBER_RADIO, "value"),
State(ID_UPLOAD_CONTENTS, "filename"),
State(ID_UPLOAD_CONTENTS, "contents"),
State(ID_LOGIN_STORE, "data"),
State(ID_SUBMISSION_TRIGGER_STORE, "data"),
prevent_initial_call=True)
def on_submission_submit_clicked(
n_submit_clicks: int,
n_confirmation_cancel_clicks: int,
n_confirmation_accept_clicks: int,
n_info_accept_clicks: int,
proj_number: int,
file_name: str,
file_contents: str,
login_store: Dict[str, Union[str, None]],
submission_trigger_store: bool):
ctx = dash.callback_context
trigger_id = ctx.triggered[0]["prop_id"].split(".")[0]
if trigger_id == ID_SUBMISSION_SUBMIT_BUTTON:
if n_submit_clicks is None:
raise PreventUpdate
# validate
error_message = on_submit_button_clicked(proj_number, file_name, file_contents, login_store)
if error_message is None: # good to go
# show the confirmation modal
return True, dash.no_update, dash.no_update, dash.no_update
else:
# show the error message in the info modal
return dash.no_update, True, error_message, dash.no_update
elif trigger_id == ID_SUBMISSION_CONFIRMATION_MODAL_CANCEL:
if n_confirmation_cancel_clicks is None:
raise PreventUpdate
# hide the confirmation modal
return False, dash.no_update, dash.no_update, dash.no_update
elif trigger_id == ID_SUBMISSION_CONFIRMATION_MODAL_ACCEPT:
if n_confirmation_accept_clicks is None:
raise PreventUpdate
# hide the confirmation modal and trigger on_submit_confirmed
return False, dash.no_update, dash.no_update, not submission_trigger_store # just flip the value, whatever it is
elif trigger_id == ID_SUBMISSION_INFO_MODAL_ACCEPT:
if n_info_accept_clicks is None:
raise PreventUpdate
# hide the info modal
return dash.no_update, False, dash.no_update, dash.no_update
else: # error
print(f"unknown button id: {trigger_id}") # TODO: better logging
return [dash.no_update] * 4 # one for each Output
@app.long_callback(
progress=[Output(ID_SUBMISSION_OUTPUT, "children")],
progress_default=[dash.no_update], # I'll set stuff manually, thank you very much
output=[Output(ID_SUBMISSION_REFRESH_DIV, "style")],
inputs=[
Input(ID_SUBMISSION_TRIGGER_STORE, "data"),
State(ID_SUBMIT_PROJECT_NUMBER_RADIO, "value"),
State(ID_UPLOAD_CONTENTS, "contents"),
State(ID_LOGIN_STORE, "data")
],
running=[ # hide the submit button when it starts running, then keep it hidden so they have to refresh to submit again
(Output(ID_SUBMISSION_SUBMIT_BUTTON, "style"), STYLE_HIDDEN, STYLE_HIDDEN),
(Output(ID_SUBMISSION_LOADING, "style"), STYLE_DIV_VISIBLE, STYLE_HIDDEN),
(Output(ID_SUBMISSION_OUTPUT, "style"), STYLE_DIV_VISIBLE_TOP_MARGIN, STYLE_DIV_VISIBLE_TOP_MARGIN)
],
prevent_initial_call=True)
def on_submit_confirmed(
set_progress: Callable,
submission_trigger_store: bool,
proj_number: int,
file_contents,
login_store):
print("start of long callback")
set_progress([None])
print("after update progress to None")
# actually run the submission
run_submission(proj_number, file_contents, login_store, set_progress)
print("finished run_submission")
# wait a sec to make sure all `set_progress` calls can finish
time.sleep(1)
# show the "clear page" message/button
print("about to return")
return [STYLE_DIV_VISIBLE_TOP_MARGIN]
@app.callback(Output(ID_SUBMISSION_ROOT_DIV, "children"),
Input(ID_SUBMISSION_REFRESH_BUTTON, "n_clicks"),
prevent_initial_call=True)
def on_submission_refresh_clicked(n_refresh_clicks: int):
# reset everything
return LAYOUT_DEFAULT_CONTENTS
环境
python版本
$ python --version
Python 3.9.6
已安装的软件包
$ pip list
Package Version
------------------------- ---------
amqp 5.0.6
billiard 3.6.4.0
Brotli 1.0.9
celery 5.1.2
click 7.1.2
click-didyoumean 0.0.3
click-plugins 1.1.1
click-repl 0.2.0
dash 2.0.0
dash-bootstrap-components 0.13.1
dash-core-components 2.0.0
dash-gif-component 1.1.0
dash-html-components 2.0.0
dash-table 5.0.0
dill 0.3.4
diskcache 5.2.1
Flask 2.0.1
Flask-Compress 1.10.1
greenlet 1.1.1
gunicorn 20.1.0
itsdangerous 2.0.1
Jinja2 3.0.1
kombu 5.1.0
MarkupSafe 2.0.1
multiprocess 0.70.12.2
orjson 3.6.3
pip 21.2.4
plotly 5.3.1
prompt-toolkit 3.0.20
psutil 5.8.0
pytz 2021.1
redis 3.5.3
setuptools 56.0.0
six 1.16.0
SQLAlchemy 1.4.23
tenacity 8.0.1
vine 5.0.0
wcwidth 0.2.5
Werkzeug 2.0.1
wheel 0.37.0
redis
$ redis-server --version
Redis server v=6.2.5 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=2a367e4b809d24de
$ redis-cli ping
PONG
$ curl localhost:6379
curl: (52) Empty reply from server
$ sudo ss -lptn 'sport = :6379'
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 511 127.0.0.1:6379 0.0.0.0:* users:(("redis-server",pid=373,fd=6))
OS 详情
Windows、WSL 2、Ubuntu
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
从 plotly 社区论坛重新发布解决方案：
https://community.plotly.com/t/long-callback-with-celery-redis-how-to-get-the-example-app-work/57663
总结
为了使长回调起作用，我需要启动 3 个协同工作的独立进程：
- Redis 服务器：
redis-server - Celery 应用程序：
celery -A app.celery worker --loglevel=INFO - Dash 应用程序：
python app.py
上面列出的命令是最简单的版本。使用的完整命令在适当修改后进一步给出。
详情
我将 celery 应用程序的声明从 src/website/long_callback_manager.py 移到了 src/app.py 以便于外部访问：
import dash
import dash_bootstrap_components as dbc
from celery import Celery
from dash.long_callback import CeleryLongCallbackManager
from website.layout_main import define_callbacks, layout
celery_app = Celery(
__name__,
broker="redis://localhost:6379/0",
backend="redis://localhost:6379/1"
)
LONG_CALLBACK_MANAGER = CeleryLongCallbackManager(celery_app)
app = dash.Dash(
__name__,
update_title="Loading...",
external_stylesheets=[
dbc.themes.BOOTSTRAP,
"https://codepen.io/chriddyp/pen/bWLwgP.css"
],
long_callback_manager=LONG_CALLBACK_MANAGER
)
app.title = "CS 236 | Project Submissions"
app.layout = layout
define_callbacks(app)
server = app.server # expose for gunicorn
if __name__ == "__main__":
app.run_server(debug=True, host="0.0.0.0")
然后我使用以下 bash 脚本来简化启动一切的过程：
#!/bin/bash
set -e # quit on any error
# make sure the redis server is running
if ! redis-cli ping > /dev/null 2>&1; then
redis-server --daemonize yes --bind 127.0.0.1
redis-cli ping > /dev/null 2>&1 # the script halts if redis is not now running (failed to start)
fi
# activate the venv that has our things installed with pip
. venv/bin/activate
# make sure it can find the python modules, but still run from this directory
export PYTHONPATH=src
# make sure we have a log directory
mkdir -p Log
# start the celery thing
celery -A app.celery_app worker --loglevel=INFO >> Log/celery_info.log 2>&1 &
# start the server
gunicorn --workers=4 --name=passoff_website_server --bind=127.0.0.1:8050 app:server >> Log/gunicorn.log 2>&1
此脚本的进程然后是 celery 和 gunicorn 子进程的父进程，并且所有这些都可以通过终止父进程作为一个包终止。