下载文件较大时，节点 FTP 不执行 `once('close')` 回调

Question

我有一种方法可以从 FTP 服务器下载文件，它可以很好地处理较小的文件，但是当我用它来下载 ~5GB 大小的 zip 类型的文件时，它会下载它，但在那之后它没有做任何事情。当下载达到 100% 时，脚本不会继续。如果下载完成后它实际上在后台执行某些操作，我应该等待吗？有文件大小限制吗？

const FTP = require('ftp')

可在 npm

上找到

downloadFile: params => {
    return new Promise((resolve, reject) => {
      let ftpClient = new FTP()
      let total = params.state.fileSize
      let progress = 0
      ftpClient.on('ready', _ => {
        console.log(`Downloading ${params.targetedFile} ...`);
        ftpClient.get(params.targetedFile, (err, stream) => {
          if (err) reject(err)
          stream.on('data', buffer => {
            progress += buffer.length
            process.stdout.write(`Progress: ${(progress/total*100).toFixed(2)}% (${progress}/${total})  \r`)
          })
          stream.once('close', _ => {
            ftpClient.end()
            console.log(`Saved downloaded file to ${params.localDir}`);
            resolve(params.localDir)
          })
          stream.pipe(fs.createWriteStream(params.localDir))
        })
      })
      ftpClient.connect(params.auth)
    })
  }

基本上，stream.once('close', ...) 的回调不会在下载大文件时执行。并且它会针对相同类型的较小文件执行。

Answer 1

我建议您处理关闭写入流的事件。

原因很简单：我们从ftp的读取流中读取并通过管道写入流，当文件成功关闭时一切正常。

所以代码：

downloadFile: params => {
    return new Promise((resolve, reject) => {
      let ftpClient = new FTP()
      let total = params.state.fileSize
      let progress = 0
      ftpClient.on('ready', _ => {
        console.log(`Downloading ${params.targetedFile} ...`);
        ftpClient.get(params.targetedFile, (err, stream) => {
          if (err) {
            ftpClient.end();
            return reject(err);
          }

          stream.on('data', buffer => {
            progress += buffer.length
            process.stdout.write(`Progress: ${(progress/total*100).toFixed(2)}% (${progress}/${total})  \r`)
          });

          // opening writeStream to file
          let finished = false;
          const writeStream = fs.createWriteStream(params.localDir);

          writeStream.on('finish', (result) => { // handling finish
            finished = true;
            ftpClient.end();
            console.log(`Finish triggered ${params.localDir}`);
            console.log(result);
            resolve(params.localDir);
          });

          writeStream.on('close', (result) => { // handling close
            ftpClient.end();
            console.log(`Close triggered ${params.localDir}`);
            console.log(result);
            resolve(params.localDir);
          })

          // piping readStream to writeStream
          stream.pipe(writeStream);
        })
      })
      ftpClient.connect(params.auth)
    })
  }

Answer 2

此代码可能会让您了解如何以有点古怪的方式处理此问题。

基本上，此方法允许您从 FTP 服务器下载文件并将其保存到本地文件系统。它在一行中输出当前进度 complete_percentage% (current/total)。完成后，它会解析 promise，returns 本地文件路径，与您作为参数传递的路径相同。

/**
   * @name downloadFile
   * @desc downloads file from FTP server
   * @param  params, Object of params
   *   @prop auth: object, or null, authorization params
   *   @prop targetedFile: {String} filename e.g. data.txt
   *   @prop localDir: {String} filename on local disk
   *   @prop state: {Object} fileinfo object, {Int} .fileSize property is required
   * @return Promise, resolves given localDir
   */
  downloadFile: params => {
    return new Promise((resolve, reject) => {
      // validate param types
      if(typeof params.auth !== 'object'
      || typeof params.targetedFile !== 'string'
      || typeof params.localDir !== 'string'
      || typeof params.state !== 'object'
      || typeof params.state.fileSize !== 'number'
      ) throw new Error('You are either missing properties or passed wrong types')

      // initialize
      let ftpClient = new FTP()
      let total = params.state.fileSize
      let progress = 0

      //
      ftpClient.on('ready', _ => {
        console.log(`Downloading ${params.targetedFile} ...`)
        // get file
        ftpClient.get(params.targetedFile, (err, stream) => {
          if (err){
            ftpClient.end()
            return reject(err)
          }

          // upon data receive
          stream.on('data', buffer => {
            progress += buffer.length
            // if progress is complete
            if(progress === total){
              // start checking if local filesize matches server filesize
              let interval = setInterval(_ => {
                if(fs.statSync(params.localDir).size === total){
                  console.log(`Downloading file complete. Location: ${params.localDir}`);
                  clearInterval(interval)
                  ftpClient.end()
                  resolve(params.localDir)
                }
              })
            }
            // show current progress in percentages and bytes
            process.stdout.write(`Progress: ${(progress/total*100).toFixed(2)}% (${progress}/${total})  \r`)
          })
          // pipe writestream to filesystem to write these bytes
          stream.pipe(fs.createWriteStream(params.localDir))
        })
      })
      ftpClient.connect(params.auth)
    })//promise
  }

下载文件较大时，节点 FTP 不执行 `once('close')` 回调

Node FTP doesn't execute `once('close')` callback when downloading file is large

javascript

ftp-client

node.js