为什么这个 FTP 块读取操作会抛出 EOFError?
Why does this FTP block read operation throw EOFError?
有人可以解释为什么这个块读取操作会抛出 EOFError 吗?在我看来,当文件被完全读取时,它只会结束块。我错过了什么?我该如何解决?非常感谢。
注意:我应该提到,有时它运行良好,有时它会抛出 EOFError。在同一位置的同一文件上...
Net::FTP.open(host=ftp.address, user='********', passwd='********') do |ftp|
ftp.passive = true
files = ftp.nlst('*')
s3 = Aws::S3::Resource.new(region: 'us-east-1')
files.each do |file|
ftp.getbinaryfile(file, file, 65536) # Copies file to /tmp
UpdateVfile.loader(@company, file) # Process file from /tmp
s3_file = file + Time.zone.now.strftime("-%Y-%m-%dT%H-%M.xls")
obj = s3.bucket('mydatafeed').object(s3_file) # Build object in S3 bucket
data = ''
ftp.getbinaryfile(file, nil, 65536) do |block| <== EOFError thrown here!
data << block
end
obj.put(body: data) # Write it to S3
ftp.delete(file) # Delete the file from FTP
end
end
堆栈跟踪:
An EOFError occurred in background at 2015.11.23 :
end of file reached
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:1112:in `readline'
-------------------------------
Backtrace:
-------------------------------
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:1112:in `readline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:289:in `getline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:300:in `getmultiline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:318:in `getresp'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:351:in `block in sendcmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:349:in `sendcmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:394:in `makepasv'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:406:in `transfercmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:490:in `block (2 levels) in retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:488:in `block in retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:487:in `retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:620:in `getbinaryfile'
/app/lib/tasks/updates_vfile.rake:40:in `block (6 levels) in <top (required)>'
FTP debug_mode 输出:
put: TYPE A
get: 200 Command TYPE okay.
put: PASV
get: 227 Entering Passive Mode (75,101,155,12,5,165)
put: NLST *
get: 150 File status okay; about to open data connection.
get: 226 Closing data connection.
put: TYPE I
get: 200 Command TYPE okay.
put: PASV
get: 227 Entering Passive Mode (75,101,155,12,5,163)
put: RETR <filename redacted>
get: 150 File status okay; about to open data connection.
get: 226 Transfer complete.
...output from update processing
put: PASV
rake aborted!
EOFError: end of file reached
问题原来是 FTP 空闲超时。因为 UpdateVfile.loader 是一项长 运行 的任务,所以我将 FTP 单独放置了太久,结果出现了 EOFError。显然,超时错误会比这种给我带来比没有诊断信息更糟糕的难以形容的故障要好得多。
如果装载机很短,它就可以工作。对于更强烈的更新,它花费的时间太长了。当我更改源以尝试诊断和解决问题时,EOFError 移至 ftp.delete 命令,这对我来说更没有意义。
我发现了一些暗示这个问题的讨论。因此,我更改了流程以在调用加载程序之前处理所有 FTP 工作。这工作可靠。否则,我要么必须定期发送保持活动状态,要么想办法修改超时期限。两者似乎都不容易实现。
如果有人能向我解释为什么 FTP 会发出 EOFError 而不是超时错误,我会很高兴。那是不对的,并且会在诊断上将任何人引向完全错误的方向。谢谢...
有人可以解释为什么这个块读取操作会抛出 EOFError 吗?在我看来,当文件被完全读取时,它只会结束块。我错过了什么?我该如何解决?非常感谢。
注意:我应该提到,有时它运行良好,有时它会抛出 EOFError。在同一位置的同一文件上...
Net::FTP.open(host=ftp.address, user='********', passwd='********') do |ftp|
ftp.passive = true
files = ftp.nlst('*')
s3 = Aws::S3::Resource.new(region: 'us-east-1')
files.each do |file|
ftp.getbinaryfile(file, file, 65536) # Copies file to /tmp
UpdateVfile.loader(@company, file) # Process file from /tmp
s3_file = file + Time.zone.now.strftime("-%Y-%m-%dT%H-%M.xls")
obj = s3.bucket('mydatafeed').object(s3_file) # Build object in S3 bucket
data = ''
ftp.getbinaryfile(file, nil, 65536) do |block| <== EOFError thrown here!
data << block
end
obj.put(body: data) # Write it to S3
ftp.delete(file) # Delete the file from FTP
end
end
堆栈跟踪:
An EOFError occurred in background at 2015.11.23 :
end of file reached
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:1112:in `readline'
-------------------------------
Backtrace:
-------------------------------
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:1112:in `readline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:289:in `getline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:300:in `getmultiline'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:318:in `getresp'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:351:in `block in sendcmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:349:in `sendcmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:394:in `makepasv'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:406:in `transfercmd'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:490:in `block (2 levels) in retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:199:in `with_binary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:488:in `block in retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:487:in `retrbinary'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/net/ftp.rb:620:in `getbinaryfile'
/app/lib/tasks/updates_vfile.rake:40:in `block (6 levels) in <top (required)>'
FTP debug_mode 输出:
put: TYPE A
get: 200 Command TYPE okay.
put: PASV
get: 227 Entering Passive Mode (75,101,155,12,5,165)
put: NLST *
get: 150 File status okay; about to open data connection.
get: 226 Closing data connection.
put: TYPE I
get: 200 Command TYPE okay.
put: PASV
get: 227 Entering Passive Mode (75,101,155,12,5,163)
put: RETR <filename redacted>
get: 150 File status okay; about to open data connection.
get: 226 Transfer complete.
...output from update processing
put: PASV
rake aborted!
EOFError: end of file reached
问题原来是 FTP 空闲超时。因为 UpdateVfile.loader 是一项长 运行 的任务,所以我将 FTP 单独放置了太久,结果出现了 EOFError。显然,超时错误会比这种给我带来比没有诊断信息更糟糕的难以形容的故障要好得多。
如果装载机很短,它就可以工作。对于更强烈的更新,它花费的时间太长了。当我更改源以尝试诊断和解决问题时,EOFError 移至 ftp.delete 命令,这对我来说更没有意义。
我发现了一些暗示这个问题的讨论。因此,我更改了流程以在调用加载程序之前处理所有 FTP 工作。这工作可靠。否则,我要么必须定期发送保持活动状态,要么想办法修改超时期限。两者似乎都不容易实现。
如果有人能向我解释为什么 FTP 会发出 EOFError 而不是超时错误,我会很高兴。那是不对的,并且会在诊断上将任何人引向完全错误的方向。谢谢...