504 网关超时错误使用 Firefox 68.9.0esr 在无头模式下使用 GeckoDriver Selenium 和 Python
504 Gateway Time-out error using Firefox 68.9.0esr in headless mode with GeckoDriver Selenium and Python
我用硒做了一个容器镜像。我使用了 RedHat 的 ubi image 作为基础图像。
我安装了以下版本的 geckodriver 和 Mozilla Firefox
- 壁虎驱动程序 0.26.0 (e9783a644016 2019-10-10 13:38 +0000)
- Mozilla Firefox 68.9.0esr
我可以 运行 在 firefox 上成功进行 selenium 测试。当我 运行 进行无头测试时，出现问题：
Traceback (most recent call last):
File "./selenium-test.py", line 58, in <module>
check_browser("FIREFOX")
File "./selenium-test.py", line 26, in check_browser
driver.get("http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console")
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 208, in check_response
raise exception_class(value)
selenium.common.exceptions.WebDriverException: Message: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
来自 OpenShift 的日志，其中容器 运行s:
09:54:45.599 INFO [ActiveSessionFactory.lambda$apply] - Matched factory org.openqa.selenium.grid.session.remote.ServicedSession$Factory (provider: org.openqa.selenium.firefox.GeckoDriverService)
Failed to open connection to "session" message bus: Address does not contain a colon
1593770085639 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileaaKtFi"
1593770085639 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:34835
Failed to open connection to "session" message bus: Address does not contain a colon
Running without a11y support!
*** You are running in headless mode.
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1593770086749 Marionette TRACE Received observer notification profile-after-change
1593770086826 Marionette TRACE Received observer notification command-line-startup
1593770086826 Marionette TRACE Received observer notification nsPref:changed
1593770086826 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1593770086938 Marionette TRACE Received observer notification toplevel-window-ready
JavaScript error: undefined, line 14: Error: An unexpected error occurred
JavaScript error: moz-extension://b2c6e410-fc73-460e-9587-3e165c5ce4ac/lib/picture_in_picture_overrides.js, line 15: Error: Incorrect argument types for pictureInPictureParent.setOverrides.
JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 1325: uncaught exception: 2147746065
1593770096320 Marionette TRACE Received observer notification marionette-startup-requested
1593770096320 Marionette TRACE Waiting until startup recorder finished recording startup scripts...
1593770096523 Marionette TRACE All scripts recorded.
1593770096524 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1593770096524 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1593770096525 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1593770096609 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1593770096610 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1593770096611 Marionette DEBUG Setting recommended pref browser.search.update to false
1593770096611 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1593770096611 Marionette DEBUG Setting recommended pref browser.tabs.remote.separatePrivilegedContentProcess to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.unloadOnLowMemory to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1593770096612 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1593770096612 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1593770096613 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1593770096613 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1593770096613 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1593770096613 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1593770096613 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1593770096613 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1593770096614 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1593770096614 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1593770096614 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1593770096614 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1593770096614 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1593770096614 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1593770096614 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1593770096615 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1593770096615 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1593770096821 Marionette INFO Listening on port 34835
1593770096821 Marionette DEBUG Remote service is active
1593770096914 Marionette DEBUG Accepted connection 0 from 127.0.0.1:47282
1593770096915 geckodriver::marionette DEBUG Connection to Marionette established on 127.0.0.1:34835.
1593770096921 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1593770096922 Marionette WARN TLS certificate errors will be ignored for this session
1593770097114 Marionette TRACE [4294967297] Frame script loaded
1593770097117 Marionette TRACE [4294967297] Frame script registered
1593770097210 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"f0d2e302-c592-49a6-bca6-4f597f778385","capabilities":{"browserName":"firefox","browserVersion":"68.9. ... /rust_mozprofileaaKtFi","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1593770097214 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"f0d2e302-c592-49a6-bca6-4f597f778385","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"68.9.0","moz:accessibilityChecks":false,"moz:buildID":"20200529103754","moz:geckodriverVersion":"0.26.0","moz:headless":true,"moz:processID":363,"moz:profile":"/tmp/rust_mozprofileaaKtFi","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.18.0-147.8.1.el8_1.x86_64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
09:54:57.215 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
09:54:57.216 INFO [RemoteSession$Factory.lambda$performHandshake[=12=]] - Started new session f0d2e302-c592-49a6-bca6-4f597f778385 (org.openqa.selenium.firefox.GeckoDriverService)
1593770102314 webdriver::server DEBUG -> POST /session/f0d2e302-c592-49a6-bca6-4f597f778385/url {"url": "http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console"}
1593770102317 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console"}]
1593770103015 Marionette TRACE [4294967297] Received DOM event beforeunload for about:blank
我尝试了官方的selenium 镜像，它使用Mozilla 75 版和相同的Geckodriver。他们利用 ubuntu 仿生。
这些 gecko 驱动程序不支持 Mozilla Firefox 68.9.0esr 吗？还是因为其他原因导致 headless 模式不起作用？
这个错误信息...
selenium.common.exceptions.WebDriverException: Message: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
...表示 Firefox v68.9.0esr 无法与应用程序服务器建立连接并引发以下 WebDriverException 错误。
504 网关超时 错误是不确定的，因为它们表明您的网站发生了什么，但它们并没有传达到底出了什么问题，因此很难分析根本原因和解决问题。但是，错误 504 Gateway Time-out 背后的原因可能有很多，解决问题的几种方法如下：
- 查找服务器连接问题：检查服务器是否因维护或任何其他原因而关闭，您的网站可能会为访问者提供 504 网关超时错误页面。解决此问题的唯一方法是等待您的服务器完成维护或修复导致错误的问题。
- 检查是否有任何 DNS 更改：如果应用程序最近更改了主机服务器或将网站移动到不同的 IP 地址网站的 DNS 服务器发生更改，可能会导致您的网站为其访问者提供 504 网关超时错误 页面。然后网站将无法启动，运行 直到 DNS 更改完全生效，这可能需要几个小时。
- 浏览日志：日志将提供有关服务器行为和状态的详细信息。分析它们以发现任何有用的信息。
- 修复错误的防火墙配置：防火墙是网站的看门人，保护您的网站免受恶意访问者或分布式拒绝服务 (DDoS) 攻击。错误的防火墙配置会导致您的防火墙将来自内容分发网络的请求视为对您的服务器的攻击并拒绝它们，从而导致 504 网关超时 错误。检查您的防火墙配置以查明并解决问题。
这个用例
但是对于这个用例，这个错误消息...
1593770085639 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:34835
Failed to open connection to "session" message bus: Address does not contain a colon
Running without a11y support!
...表示 Firefox v68.9.0esr 构建存在问题。
解决方案
根据 comment within the discussion (firefox:28877): GConf-WARNING **: Client failed to connect to the D-BUS daemon: Address does not contain a colon 解决方案是恢复到以前版本的 Firefox ESR 构建。
我用硒做了一个容器镜像。我使用了 RedHat 的 ubi image 作为基础图像。
我安装了以下版本的 geckodriver 和 Mozilla Firefox
- 壁虎驱动程序 0.26.0 (e9783a644016 2019-10-10 13:38 +0000)
- Mozilla Firefox 68.9.0esr
我可以 运行 在 firefox 上成功进行 selenium 测试。当我 运行 进行无头测试时，出现问题：
Traceback (most recent call last):
File "./selenium-test.py", line 58, in <module>
check_browser("FIREFOX")
File "./selenium-test.py", line 26, in check_browser
driver.get("http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console")
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 208, in check_response
raise exception_class(value)
selenium.common.exceptions.WebDriverException: Message: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
来自 OpenShift 的日志，其中容器 运行s:
09:54:45.599 INFO [ActiveSessionFactory.lambda$apply] - Matched factory org.openqa.selenium.grid.session.remote.ServicedSession$Factory (provider: org.openqa.selenium.firefox.GeckoDriverService)
Failed to open connection to "session" message bus: Address does not contain a colon
1593770085639 mozrunner::runner INFO Running command: "/usr/bin/firefox" "-marionette" "-headless" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileaaKtFi"
1593770085639 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:34835
Failed to open connection to "session" message bus: Address does not contain a colon
Running without a11y support!
*** You are running in headless mode.
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1593770086616 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1593770086749 Marionette TRACE Received observer notification profile-after-change
1593770086826 Marionette TRACE Received observer notification command-line-startup
1593770086826 Marionette TRACE Received observer notification nsPref:changed
1593770086826 Marionette DEBUG Init aborted (running=false, enabled=true, finalUIStartup=false)
1593770086938 Marionette TRACE Received observer notification toplevel-window-ready
JavaScript error: undefined, line 14: Error: An unexpected error occurred
JavaScript error: moz-extension://b2c6e410-fc73-460e-9587-3e165c5ce4ac/lib/picture_in_picture_overrides.js, line 15: Error: Incorrect argument types for pictureInPictureParent.setOverrides.
JavaScript error: resource:///modules/sessionstore/SessionStore.jsm, line 1325: uncaught exception: 2147746065
1593770096320 Marionette TRACE Received observer notification marionette-startup-requested
1593770096320 Marionette TRACE Waiting until startup recorder finished recording startup scripts...
1593770096523 Marionette TRACE All scripts recorded.
1593770096524 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000
1593770096524 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99
1593770096525 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true
1593770096609 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false
1593770096610 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true
1593770096611 Marionette DEBUG Setting recommended pref browser.search.update to false
1593770096611 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false
1593770096611 Marionette DEBUG Setting recommended pref browser.tabs.remote.separatePrivilegedContentProcess to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.unloadOnLowMemory to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false
1593770096612 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false
1593770096612 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to
1593770096612 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false
1593770096613 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false
1593770096613 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true
1593770096613 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false
1593770096613 Marionette DEBUG Setting recommended pref dom.file.createInChild to true
1593770096613 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0
1593770096613 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0
1593770096614 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false
1593770096614 Marionette DEBUG Setting recommended pref extensions.webservice.discoverURL to http://%(server)s/dummy/discoveryURL
1593770096614 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false
1593770096614 Marionette DEBUG Setting recommended pref network.http.speculative-parallel-limit to 0
1593770096614 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false
1593770096614 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0
1593770096614 Marionette DEBUG Setting recommended pref signon.autofillForms to false
1593770096615 Marionette DEBUG Setting recommended pref signon.rememberSignons to false
1593770096615 Marionette DEBUG Setting recommended pref toolkit.cosmeticAnimations.enabled to false
1593770096821 Marionette INFO Listening on port 34835
1593770096821 Marionette DEBUG Remote service is active
1593770096914 Marionette DEBUG Accepted connection 0 from 127.0.0.1:47282
1593770096915 geckodriver::marionette DEBUG Connection to Marionette established on 127.0.0.1:34835.
1593770096921 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}]
1593770096922 Marionette WARN TLS certificate errors will be ignored for this session
1593770097114 Marionette TRACE [4294967297] Frame script loaded
1593770097117 Marionette TRACE [4294967297] Frame script registered
1593770097210 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"f0d2e302-c592-49a6-bca6-4f597f778385","capabilities":{"browserName":"firefox","browserVersion":"68.9. ... /rust_mozprofileaaKtFi","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
1593770097214 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"f0d2e302-c592-49a6-bca6-4f597f778385","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"68.9.0","moz:accessibilityChecks":false,"moz:buildID":"20200529103754","moz:geckodriverVersion":"0.26.0","moz:headless":true,"moz:processID":363,"moz:profile":"/tmp/rust_mozprofileaaKtFi","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.18.0-147.8.1.el8_1.x86_64","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
09:54:57.215 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
09:54:57.216 INFO [RemoteSession$Factory.lambda$performHandshake[=12=]] - Started new session f0d2e302-c592-49a6-bca6-4f597f778385 (org.openqa.selenium.firefox.GeckoDriverService)
1593770102314 webdriver::server DEBUG -> POST /session/f0d2e302-c592-49a6-bca6-4f597f778385/url {"url": "http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console"}
1593770102317 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"http://selenium-hub-cpet-selenium.apps.idc.ont.belastingdienst.nl/grid/console"}]
1593770103015 Marionette TRACE [4294967297] Received DOM event beforeunload for about:blank
我尝试了官方的selenium 镜像，它使用Mozilla 75 版和相同的Geckodriver。他们利用 ubuntu 仿生。
这些 gecko 驱动程序不支持 Mozilla Firefox 68.9.0esr 吗？还是因为其他原因导致 headless 模式不起作用？
这个错误信息...
selenium.common.exceptions.WebDriverException: Message: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
...表示 Firefox v68.9.0esr 无法与应用程序服务器建立连接并引发以下 WebDriverException 错误。
504 网关超时 错误是不确定的，因为它们表明您的网站发生了什么，但它们并没有传达到底出了什么问题，因此很难分析根本原因和解决问题。但是，错误 504 Gateway Time-out 背后的原因可能有很多，解决问题的几种方法如下：
- 查找服务器连接问题：检查服务器是否因维护或任何其他原因而关闭，您的网站可能会为访问者提供 504 网关超时错误页面。解决此问题的唯一方法是等待您的服务器完成维护或修复导致错误的问题。
- 检查是否有任何 DNS 更改：如果应用程序最近更改了主机服务器或将网站移动到不同的 IP 地址网站的 DNS 服务器发生更改，可能会导致您的网站为其访问者提供 504 网关超时错误 页面。然后网站将无法启动，运行 直到 DNS 更改完全生效，这可能需要几个小时。
- 浏览日志：日志将提供有关服务器行为和状态的详细信息。分析它们以发现任何有用的信息。
- 修复错误的防火墙配置：防火墙是网站的看门人，保护您的网站免受恶意访问者或分布式拒绝服务 (DDoS) 攻击。错误的防火墙配置会导致您的防火墙将来自内容分发网络的请求视为对您的服务器的攻击并拒绝它们，从而导致 504 网关超时 错误。检查您的防火墙配置以查明并解决问题。
这个用例
但是对于这个用例，这个错误消息...
1593770085639 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:34835
Failed to open connection to "session" message bus: Address does not contain a colon
Running without a11y support!
...表示 Firefox v68.9.0esr 构建存在问题。
解决方案
根据 comment within the discussion (firefox:28877): GConf-WARNING **: Client failed to connect to the D-BUS daemon: Address does not contain a colon 解决方案是恢复到以前版本的 Firefox ESR 构建。