NodeJS ssh2 fastGet解压挂起

Question

使用 mscdex 的 nodejs ssh2 库，我有一个 sftp "fetch" 节点脚本，它将列出 ssh 服务器目录的内容，搜索特定文件，然后下载它们。当脚本到达调用 "sftp.fastGet" 的下载步骤时，永远不会调用提供给 fastGet 的回调。

我已经打开了 ssh 连接选项中的 "DEBUG" 设置。似乎 sftp.fastGet 正在执行，但挂在 "decompress" 步骤上。这是我得到的输出：

...
DEBUG: Parser: Verifying MAC
DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
DEBUG: Parser: Decompressing
// hangs here forever (or until timeout)...

更奇怪的是，好几个数据包都解压过来了。这是（更详细的）调试：

info: Fetching xxx from /xxx/xxx/xxx to X:\xxx\xxx\xxx
debug: DEBUG[SFTP]: Outgoing: Writing OPEN
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG: Parser: IN_PACKET
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: pktLen:20,padLen:5,remainLen:16
debug: DEBUG: Parser: IN_PACKETDATA
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: HMAC size:20
debug: DEBUG: Parser: IN_PACKETDATAVERIFY
debug: DEBUG: Parser: Verifying MAC
debug: DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
debug: DEBUG: Parser: Decompressing
// doesn't seem to hang here
debug: DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Parser: Response: HANDLE
debug: DEBUG[SFTP]: Outgoing: Writing FSTAT
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG: Parser: IN_PACKETBEFORE (expecting 8)
debug: DEBUG: Parser: IN_PACKET
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: pktLen:28,padLen:10,remainLen:24
debug: DEBUG: Parser: IN_PACKETDATA
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: HMAC size:20
debug: DEBUG: Parser: IN_PACKETDATAVERIFY
debug: DEBUG: Parser: Verifying MAC
debug: DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
debug: DEBUG: Parser: Decompressing
// doesn't seem to hang here
debug: DEBUG: Parser: IN_PACKETDATAAFTER, packet: CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Parser: Response: ATTRS
debug: DEBUG: Parser: IN_PACKETBEFORE (expecting 8)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG[SFTP]: Outgoing: Writing READ
debug: DEBUG: Outgoing: Writing CHANNEL_DATA (0)
debug: DEBUG: Parser: IN_PACKET
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: pktLen:2388,padLen:6,remainLen:2384
debug: DEBUG: Parser: IN_PACKETDATA
debug: DEBUG: Parser: Decrypting
debug: DEBUG: Parser: HMAC size:20
debug: DEBUG: Parser: IN_PACKETDATAVERIFY
debug: DEBUG: Parser: Verifying MAC
debug: DEBUG: Parser: IN_PACKETDATAVERIFY (Valid HMAC)
debug: DEBUG: Parser: Decompressing
// hangs here forever...

破解打开“[我的项目]\node_modules\ssh2-streams\lib\ssh.js”，发现"decompress.instance.flush"方法回调并不是每次都被调用

ssh.js line 544: ...
} else if (instate.status === IN_PACKETDATAAFTER) {
  if (decompress.instance) {
  if (!decomp) {
    debug('DEBUG: Parser: Decompressing');
    decompress.instance.write(instate.payload);
    // this function executes and calls the method below
    decompress.instance.flush(Z_PARTIAL_FLUSH, function(){
      // this callback function is called during the first two iterations,
      // but is not called the last time, when the process hangs
      instate.payload = decompress.instance.read();
      var nextSlice;
      if (i === chlen)
        nextSlice = EMPTY_BUFFER;
      else
        nextSlice = chunk.slice(i);
      self._transform(nextSlice, encoding, callback, true);
    });
    return;
  } else {
  ...

...当然，这是我的代码运行

var sshClient = require('ssh2').Client;

var client = new sshClient();

client.on('ready', ()=> {

  client.sftp( (sftpErr, sftp) => {

    sftp.readdir(remotepath, (dirErr, files) => {

      var validFiles = files.filter( (file) => {
        return file.filename.match(regex);
      });  

      async.eachSeries(validFiles, (ftpFile, cb) => {

        var remote = remotepath + ftpFile.filename;
        var local = path.join(localpath, ftpFile.filename);


        console.log('Fetching ' + ftpFile.filename + ' from ' + remote + ' to ' + local);

        sftp.fastGet(remote, local, (getErr) => {
          console.log('Fast Get Complete');
          // this is never called
        });

      });

    })

  });

});

client.connect({
  host: "xxx.xxx.xxx.xxx",
  port: 22,
  username: "someuser",
  password: "somepass",
  debug: console.log,
  algorithms: {
    key: [
      "diffie-hellman-group1-sha1",
    ],
    cipher: [
      "blowfish-cbc",
      "3des-cbc"
    ],
    compress: [
      "zlib"
    ],
    hmac: [
      "hmac-sha1",
      "hmac-md5"
    ]
  }
});

Answer 1

只有在从 zlib 流中读取所有（当前）未压缩数据后，才会调用 flush() 回调。这只是当未压缩的数据长度超过 zlib 流的 highWaterMark 时的问题。现在应该从 mscdex/ssh2-streams@8c0906c6b8.

开始修复此问题

NodeJS ssh2 fastGet解压挂起

NodeJS ssh2 fastGet hangs on decompression

compression

ssh

sftp

freeze

node.js