Encoding::UndefinedConversionError 从 Rails 应用程序调用 Chef 更新命令时

Encoding::UndefinedConversionError when calling Chef update command from a Rails application

我正在构建一个使用 Chef-DK 底层的 Rails 应用程序。

我想要实现的是执行与 chef update <path/to/policyfile>.

相同的代码

这是代码中有趣的部分:

require 'chef-dk/command/update'

chef_update = ChefDK::Command::Update.new
update_options = [policyfile_path]
update_options << '--debug' if Rails.env.development?
update = chef_update.run(update_options)

当此 运行 时,它开始更新策略文件但失败并显示以下输出:

Attributes already up to date
Building policy k8sworker
Expanded run list: recipe[fail2ban], recipe[os-hardening], recipe[rkhunter], recipe[ssh-hardening], recipe[ssh-keys], recipe[tincvpn]
Caching Cookbooks...
Installing tincvpn ~> 0.1.10 from github
Installing fail2ban 6.0.0
Error: Failed to generate Policyfile.lock
Reason: (Encoding::UndefinedConversionError) "\x8B" from ASCII-8BIT to UTF-8
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `write'
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `block in delegating_block'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:528:in `block in stream_to_tempfile'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:497:in `call_block'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:488:in `<<'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:163:in `read'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:293:in `block in read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:253:in `inflater'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:527:in `stream_to_tempfile'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:230:in `block in streaming_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:92:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1518:in `block in transport_request'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:165:in `reading_body'
/usr/local/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1479:in `request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1472:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:920:in `start'
/usr/local/lib/ruby/2.6.0/net/http.rb:1470:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:69:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:370:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:388:in `block (2 levels) in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:485:in `follow_redirect'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:383:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:228:in `streaming_request'
/usr/local/bundle/gems/cookbook-omnifetch-0.8.1/lib/cookbook-omnifetch/artifactserver.rb:42:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile/cookbook_location_specification.rb:81:in `ensure_cached'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:181:in `block in install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `each'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:101:in `generate_lock_and_install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:63:in `run'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/command/update.rb:93:in `run'
/application/app/interactors/node_actions/run_chef_update.rb:24:in `call'
...

这是 trace 日志级别的 Chef 输出:

worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Initiating GET to https://supermarket.chef.io/universe
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: cache-control: max-age=0, private, must-revalidate
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-encoding: gzip
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-type: application/json; charset=utf-8
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:19 GMT
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: etag: W/"461b5134d1467f6950f82af1510078a0"
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-request-id: 0bc2cc4a-0399-4953-93db-699176e11e36
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-runtime: 0.301233
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: transfer-encoding: chunked
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Decompressing gzip response
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Initiating GET to https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 302 Found
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: cache-control: no-cache
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: text/html; charset=utf-8
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:25 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: location: https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-request-id: 17a0d07d-cf03-45b3-956f-8eb7fcabd2a7
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-runtime: 0.044161
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 209
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Response Body ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: <html><body>You are being <a href="https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785">redirected</a>.</body></html>
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Response Body -----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: No content-length information collected for the streamed download, cannot identify streamed download.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Following redirect 1/10
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Initiating GET to https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-id-2: a2iOnve5iX8paa/pT2s+A21Bm8IvSqbjzYeP8mc2I2kNGSwgj9Wen8IyQlpiQ955grAF7xZKaQE=
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-request-id: E74D7A5F59B4B3B0
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:26 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: last-modified: Wed, 08 May 2019 21:59:47 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: etag: "5c15105980cf9bfb2467ed00b789be0a"
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: accept-ranges: bytes
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: application/gzip
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 9020
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: AmazonS3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Streaming download from https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download to tempfile /tmp/chef-rest20190703-1-cbdl7x
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content_encoding = ''               initializing noop stream deflator.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk

这就像从 Amazon S3 压缩 fail2ban 存档时,由于某些编码问题而失败。

从那时起:

  1. 当我然后从终端运行 chef update 命令时,它工作正常,没有错误
  2. 当我再次 运行 我的代码时,它也可以正常工作,没有任何错误(我猜是因为厨师缓存了食谱的版本,所以不再检查失败的代码)

所以对我来说,为了使这部分正常工作,我的代码中好像缺少了某些东西,但我不知道它会是什么。

我该如何解决这个问题?

P.S.: 该应用程序 运行 与 Linux Alpine 一起安装在 Docker 容器中。

Chef 库正在使用 Tempfile 以便将下载的数据写入某处,因此我为其 write 方法添加了别名以便在传递参数之前调用 force_encoding('UTF-8')它回到原来的 write 方法。

这是我的覆盖(在 config/initializers/tempfile_override.rb :

# frozen_string_literal: true

require 'tempfile'

#
# Overrides the Tempfile write method in order to always use `force_encode` with
# UTF-8 in order to solve the issue where `chef update` fails to write
# downloaded cookbooks data and write it to the disk.
# See 
#
class Tempfile
  alias old_write write
  def write(data)
    old_write(data.force_encoding('UTF-8'))
  end
end

我有点被迫这样做,因为我需要更改 Chef::UI 对象以便在我的应用程序中获取 Chef 日志。

这并不干净,但我没有更好的方法来做到这一点而不必修补厨师本身。

在找到更好的答案之前,此答案将被接受。

如果你知道更好的方法,请帮助我here :)