golang tar.Read 截断数据

Question

我在使用 golang tar 库提取 tar 存档时遇到问题。 header 读取正确的大小，我的缓冲区大小相同，但是当我调用 tar.Read 时，我没有获得缓冲区中的所有数据。我不明白为什么会这样。

tar球最多包含 4 个文件，似乎是 thermostat.csv 文件被截断了。

当我使用 bash 提取文件时，它们是干净的，所以我很确定 tar 球没有问题。

package main

import (
    "archive/tar"
    "compress/gzip"
    "os"
    "io"
    "io/ioutil"
    "fmt"
)

func main() {
    var filename string
    //untar each file
    filename = "/tmp/6yHYKND3t0"
    f,err := os.Open(filename)
    if err != nil {
        panic("cant open file")
    }
    defer f.Close()
    gzf,err := gzip.NewReader(f)
    check(err)
    tarReader := tar.NewReader(gzf)
    for {
        header,err := tarReader.Next()
        if err == io.EOF {
            break
        }
        check(err)
        name := header.Name
        switch header.Typeflag {
            case tar.TypeDir:
                os.Mkdir("/tmp/enTest/"+name,0755)
            case tar.TypeReg:
                data := make([]byte,header.Size)
                fmt.Printf("Header Size: %d\n",header.Size)
                l,err := tarReader.Read(data)
                fmt.Printf("Length of buffer: %d\n", len(data))
                fmt.Printf("Read from tar.Read: %d\n",l)
                check(err)
                outputFile := "/tmp/enTest/"+name
                fmt.Println("Writing file to :",outputFile)
                ioutil.WriteFile(outputFile,data,0755)
            default:
                fmt.Println("Unknown type %c %s\n",header.Typeflag,name)
                panic(err)
        }
    }
}

func check(err error) {
    if err != nil {
        panic(err)
    }
}

输出如下：

Header Size: 47537370
Length of buffer: 47537370
Read from tar.Read: 31744
Writing file to : /tmp/enTest/d2af74605ac45cf6b19c068d2d8e2710/thermostats.csv
Header Size: 34038106
Length of buffer: 34038106
Read from tar.Read: 7168
Writing file to : /tmp/enTest/d2af74605ac45cf6b19c068d2d8e2710/energy-monitors.csv
Header Size: 72965
Length of buffer: 72965
Read from tar.Read: 14336
Writing file to : /tmp/enTest/d2af74605ac45cf6b19c068d2d8e2710/lights.csv
Header Size: 7158655
Length of buffer: 7158655
Read from tar.Read: 6144
Writing file to : /tmp/enTest/d2af74605ac45cf6b19c068d2d8e2710/remote-sensors.csv
Header Size: 1732617
Length of buffer: 1732617
Read from tar.Read: 23040
Writing file to : /tmp/enTest/d2af74605ac45cf6b19c068d2d8e2710/hvac-controllers.cs

v

Answer 1

io.Reader documentation 说：

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. ... If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

读取不保证填充数据参数。使用 io.ReadFull 填充缓冲区。

l,err := io.ReadFull(tarReader, data)

golang tar.Read 截断数据

golang tar.Read truncating data

tar

go