尝试通过 Twilio 发送消息时出现奇怪的错误
Getting weird errors when trying to send a message through Twilio
我们昨天对后端进行了重大升级，现在我们遇到了一些奇怪的错误，我不知道它们是什么。这些错误似乎是在尝试通过 Twilio 的客户端发送消息时发生的。此外，我们认为由于这些错误，事情正在我们的 sidekiq 队列中备份。我们的 sidekiq 忙碌选项卡有一些东西在那里等着。以下是我们看到的一些错误：
- IOError: 关闭流
- OpenSSL::SSL::SSLError: SSL_connect 返回=1 errno=0 state=SSLv3
读取服务器问候 A：未知警报类型
- Net::ReadTimeout: Net::ReadTimeout
- EOFError: 到达文件末尾
- IOError：尝试从块中读取正文
- Errno::EBADF：错误的文件描述符
这些在过去很少发生在我们身上，现在它们发生得非常频繁。这是 IOError 的堆栈跟踪：关闭流错误：
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:155 in "select"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:155 in "rescue in rbuf_fill"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:152 in "rbuf_fill"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:134 in "readuntil"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:144 in "readline"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http/response.rb:39 in "read_status_line"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http/response.rb:28 in "read_new"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1408 in "block in transport_request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1405 in "catch"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1405 in "transport_request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1378 in "request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:27 in "block (2 levels) in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent.rb:454 in "disable_all_tracing"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:26 in "block in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/cross_app_tracing.rb:38 in "tl_trace_http_request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:23 in "request_with_newrelic_trace"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1371 in "block in request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:853 in "start"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1369 in "request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:27 in "block (2 levels) in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent.rb:454 in "disable_all_tracing"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:26 in "block in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/cross_app_tracing.rb:43 in "tl_trace_http_request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:23 in "request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/client.rb:216 in "connect_and_send"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/client.rb:168 in "block (2 levels) in <class:Client>"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/list_resource.rb:91 in "create"
/app/models/message_sender.rb:142 in "send_message_with_twilio!"
/app/models/message_sender.rb:66 in "send_sms"
/app/models/message_sender.rb:37 in "send_now"
/app/models/message_sender.rb:19 in "block (2 levels) in save_and_send_from_job"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:201 in "block in transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:209 in "within_new_transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:201 in "transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/transactions.rb:208 in "transaction"
/app/models/message_sender.rb:17 in "block in save_and_send_from_job"
/app/models/message_sender.rb:16 in "each"
/app/models/message_sender.rb:16 in "save_and_send_from_job"
/app/workers/send_single_sms_worker.rb:8 in "perform"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:49 in "block (3 levels) in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:122 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:122 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/sidekiq.rb:33 in "block in call"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:365 in "perform_action_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/sidekiq.rb:29 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidetiq-0.6.1/lib/sidetiq/middleware/history.rb:8 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-history-0.0.1/lib/sidekiq/history/middleware.rb:26 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/active_record.rb:6 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/retry_jobs.rb:62 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/bundler/gems/sidekiq-failures-485c6e56e306/lib/sidekiq/failures/middleware.rb:9 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/logging.rb:11 in "block in call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/logging.rb:22 in "with_context"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/logging.rb:7 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:127 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:127 in "invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:48 in "block (2 levels) in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:108 in "stats"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:47 in "block in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:86 in "do_defer"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:37 in "process"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26 in "public_send"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26 in "dispatch"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122 in "dispatch"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71 in "block in task"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357 in "block in task"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57 in "block in initialize"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15 in "block in create"
很可能发现了这个问题，因为自从以这种方式解决问题后我们没有遇到任何问题。原来 Twilio Client 不是线程安全的，我们试图用一堆不同的线程来锤击同一个@client。解决方法是确保我们每次需要时都创建一个新的 twilio 客户端。
我们昨天对后端进行了重大升级，现在我们遇到了一些奇怪的错误，我不知道它们是什么。这些错误似乎是在尝试通过 Twilio 的客户端发送消息时发生的。此外，我们认为由于这些错误，事情正在我们的 sidekiq 队列中备份。我们的 sidekiq 忙碌选项卡有一些东西在那里等着。以下是我们看到的一些错误：
- IOError: 关闭流
- OpenSSL::SSL::SSLError: SSL_connect 返回=1 errno=0 state=SSLv3 读取服务器问候 A：未知警报类型
- Net::ReadTimeout: Net::ReadTimeout
- EOFError: 到达文件末尾
- IOError：尝试从块中读取正文
- Errno::EBADF：错误的文件描述符
这些在过去很少发生在我们身上，现在它们发生得非常频繁。这是 IOError 的堆栈跟踪：关闭流错误：
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:155 in "select"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:155 in "rescue in rbuf_fill"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:152 in "rbuf_fill"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:134 in "readuntil"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/protocol.rb:144 in "readline"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http/response.rb:39 in "read_status_line"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http/response.rb:28 in "read_new"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1408 in "block in transport_request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1405 in "catch"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1405 in "transport_request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1378 in "request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:27 in "block (2 levels) in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent.rb:454 in "disable_all_tracing"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:26 in "block in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/cross_app_tracing.rb:38 in "tl_trace_http_request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:23 in "request_with_newrelic_trace"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1371 in "block in request"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:853 in "start"
/vendor/ruby-2.1.0/lib/ruby/2.1.0/net/http.rb:1369 in "request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:27 in "block (2 levels) in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent.rb:454 in "disable_all_tracing"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:26 in "block in request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/cross_app_tracing.rb:43 in "tl_trace_http_request"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/net.rb:23 in "request_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/client.rb:216 in "connect_and_send"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/client.rb:168 in "block (2 levels) in <class:Client>"
/vendor/bundle/ruby/2.1.0/gems/twilio-ruby-3.11.5/lib/twilio-ruby/rest/list_resource.rb:91 in "create"
/app/models/message_sender.rb:142 in "send_message_with_twilio!"
/app/models/message_sender.rb:66 in "send_sms"
/app/models/message_sender.rb:37 in "send_now"
/app/models/message_sender.rb:19 in "block (2 levels) in save_and_send_from_job"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:201 in "block in transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:209 in "within_new_transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:201 in "transaction"
/vendor/bundle/ruby/2.1.0/gems/activerecord-4.1.6/lib/active_record/transactions.rb:208 in "transaction"
/app/models/message_sender.rb:17 in "block in save_and_send_from_job"
/app/models/message_sender.rb:16 in "each"
/app/models/message_sender.rb:16 in "save_and_send_from_job"
/app/workers/send_single_sms_worker.rb:8 in "perform"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:49 in "block (3 levels) in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:122 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:122 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/sidekiq.rb:33 in "block in call"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:365 in "perform_action_with_newrelic_trace"
/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.5.251/lib/new_relic/agent/instrumentation/sidekiq.rb:29 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidetiq-0.6.1/lib/sidetiq/middleware/history.rb:8 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-history-0.0.1/lib/sidekiq/history/middleware.rb:26 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/active_record.rb:6 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/retry_jobs.rb:62 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/bundler/gems/sidekiq-failures-485c6e56e306/lib/sidekiq/failures/middleware.rb:9 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/logging.rb:11 in "block in call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/logging.rb:22 in "with_context"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/server/logging.rb:7 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:124 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:127 in "call"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/middleware/chain.rb:127 in "invoke"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:48 in "block (2 levels) in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:108 in "stats"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:47 in "block in process"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:86 in "do_defer"
/vendor/bundle/ruby/2.1.0/gems/sidekiq-2.17.7/lib/sidekiq/processor.rb:37 in "process"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26 in "public_send"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26 in "dispatch"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122 in "dispatch"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60 in "block in invoke"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71 in "block in task"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357 in "block in task"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57 in "block in initialize"
/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15 in "block in create"
很可能发现了这个问题，因为自从以这种方式解决问题后我们没有遇到任何问题。原来 Twilio Client 不是线程安全的，我们试图用一堆不同的线程来锤击同一个@client。解决方法是确保我们每次需要时都创建一个新的 twilio 客户端。