base64 解码器（io.Reader 实现）不当行为

Question

我已经尝试在 for 循环中 re-declare/assign base64 解码器并使用 os.Seek 函数返回到循环结束时的文件开头，为了使被调用的函数（在此测试用例 PrintBytes 中）能够在整个 for 循环中从头到尾一次又一次地处理文件。

这是我的（我敢肯定非常不惯用的）代码，在 main( ):

package main

import (
    "encoding/base64"
    "io"
    "log"
    "net/http"
    "os"
)

var (
    remote_file string = "http://cryptopals.com/static/challenge-data/6.txt"
    local_file  string = "secrets_01_06.txt"
)

func main() {
    f, err := os.Open(local_file)
    if err != nil {
        DownloadFile(local_file, remote_file)
        f, err = os.Open(local_file)
        if err != nil {
            log.Fatal(err)
        }
    }
    defer f.Close()

    for blocksize := 1; blocksize <= 5; blocksize++ {
        decoder := base64.NewDecoder(base64.StdEncoding, f)
        PrintBytes(decoder, blocksize)
        _, err := f.Seek(0, 0)
        if err != nil {
            log.Fatal(err)
        }
    }
}

func PrintBytes(reader io.Reader, blocksize int) {
    block := make([]byte, blocksize)
    for {
        n, err := reader.Read(block)
        if err != nil && err != io.EOF {
            log.Fatal(err)
        }
        if n != blocksize {
            log.Printf("n=%d\tblocksize=%d\tbreaking...", n, blocksize)
            break
        }
        log.Printf("%x\tblocksize=%d", block, blocksize)
    }
}

func DownloadFile(local string, url string) {
    f, err := os.Create(local)
    if err != nil {
        log.Fatal(err)
    }
    defer f.Close()

    resp, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    _, err = io.Copy(f, resp.Body)
    if err != nil {
        log.Fatal(err)
    }
}

可以在此处查看此代码的输出https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085

我不明白的是这种行为： https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085#file-bad_reader_log-L5758

我原以为它只是一次将文件从头到尾读取 2 个字节到 2 字节的切片中。为什么这里只读取了1个字节？

Answer 1

不是encoding/base64的问题。使用 io.Reader 时，不能保证读取的字节数完全等于缓冲区大小（即示例代码中的 blocksize）。文档指出：

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

在您的示例中，将 PrintBytes 更改为

func PrintBytes(reader io.Reader, blocksize int) {
    block := make([]byte, blocksize)
    for {
        n, err := reader.Read(block)
        //Process the data if n > 0, even when err != nil
        if n > 0 {
            log.Printf("%x\tblocksize=%d", block[:n], blocksize)
        }

        //Check for error
        if err != nil {
            if err != io.EOF {
                log.Fatal(err)
            } else if err == io.EOF {
                break
            }
        } else if n == 0 {
            //Considered as nothing happened
            log.Printf("WARNING: read return 0,nil")
        }
    }
}

更新：

正确使用 io.Reader，修改代码以在 n > 0 时始终处理数据，即使发生错误也是如此。

base64 解码器（io.Reader 实现）不当行为

base64 decoder (io.Reader implementation) misbehaviour

io

file

go

reader