base64 编码从 dos header 中删除回车 return

Question

我一直在尝试将应用程序的二进制数据编码为 base64（特别是提升 base64），但我运行遇到了一个问题，其中回车 return 在 dos header 编码不正确。

它应该是这样的：

This program cannot be run in DOS mode.[CR]
[CR][LF]

而是它的输出是这样的：

This program cannot be run in DOS mode.[CR][LF]

似乎第一个回车 return 被跳过，这导致 DOS header 在尝试运行程序时无效。

我正在使用的 base64 算法的代码可以在以下位置找到：https://www.boost.org/doc/libs/1_66_0/boost/beast/core/detail/base64.hpp

非常感谢！

void load_file(const char* filename, char** file_out, size_t& size_out)
{
        FILE* file;
        fopen_s(&file, filename, "r");
        if (!file)
            return false;

        fseek(file, 0, SEEK_END);
        size = ftell(file);
        rewind(file);

        *out = new char[size];
        fread(*out, size, 1, file);
        fclose(file);
}

void some_func()
{
    char* file_in;
    size_t file_in_size;
    load_file("filename.bin", &file_in, file_in_size);
    auto encoded_size = base64::encoded_size(file_in_size);
    auto file_encoded = new char[encoded_size];
    memset(0, file_encoded, encoded_size);
    base64::encode(file_encoded, file_in, file_in_size);
    std::ofstream orig("orig.bin", std::ios_base::binary);
    for (int i = 0; i < file_in_size; i++)
    {
        auto c = file_in[i];
        orig << c; // DOS header contains a NULL as the 3rd char, don't allow it to be null terminated early, may cause ending nulls but does not affect binary files.
    }
    orig.close();
    std::ofstream encoded("encoded.txt"); //pass this output through a base64 to file website.
    encoded << file_encoded; // for loop not required, does not contain nulls (besides ending null) will contain trailing encoded nulls.
   encoded.close();
   auto decoded_size = base64::decoded_size(encoded_size);
   auto file_decoded = new char[decoded_size];
   memset(0, file_decoded, decoded_size); // again trailing nulls but it doesn't matter for binary file operation. just wasted disk space.
   base64::decode(file_decoded, file_encoded, encoded_size);
   std::ofstream decoded("decoded.bin", std::ios_base::binary);
   for (int i = 0; i < decoded_size; i++)
   {
        auto c = file_decoded[i];
        decoded << c;
   }
   decoded.close();
   free(file_in);
   free(file_encoded);
   free(file_decoded);
}

上面的代码会显示读取文件不会去掉回车return，而文件编码成base64会去掉。

Answer 1

好的，感谢您添加代码！

我试过了，确实有“奇怪”，即使我简化了代码（主要是让它成为C++，而不是C）。

那你是做什么的？您查看功能的文档。这看起来很复杂，因为根据定义，毕竟 detail::base64 不是 public API 的一部分，并且“未记录”。

不过，还是可以看看涉及到的函数的注释，写的很清楚：

/** Encode a series of octets as a padded, base64 string.

    The resulting string will not be null terminated.

    @par Requires

    The memory pointed to by `out` points to valid memory
    of at least `encoded_size(len)` bytes.

    @return The number of characters written to `out`. This
    will exclude any null termination.
*/
std::size_t
encode(void* dest, void const* src, std::size_t len)

和

/** Decode a padded base64 string into a series of octets.

    @par Requires

    The memory pointed to by `out` points to valid memory
    of at least `decoded_size(len)` bytes.

    @return The number of octets written to `out`, and
    the number of characters read from the input string,
    expressed as a pair.
*/
std::pair<std::size_t, std::size_t>
decode(void* dest, char const* src, std::size_t len)

结论：错在哪里？

与“dos headers”或“carriage returns”无关。也许 fopen (what's the differences between r and rb in fopen) 中关于 "rb" 的一些东西，但为什么还要使用它：

template <typename Out> Out load_file(std::string const& filename, Out out) {
    std::ifstream ifs(filename, std::ios::binary); // or "rb" on your fopen
    ifs.exceptions(std::ios::failbit |
                   std::ios::badbit); // we prefer exceptions
    return std::copy(std::istreambuf_iterator<char>(ifs), {}, out);
}

真正的问题是：您的代码忽略了来自 encode/decode.

的所有 return 值

encoded_size 和 decoded_size 值是 估计值 ，这会给你足够 space存储结果，但您必须在执行 encoding/decoding.

后将其更正为 实际大小

这是我固定和简化的例子。注意 md5sums 的结帐方式：

Live On Coliru

#include <boost/beast/core/detail/base64.hpp>
#include <fstream>
#include <iostream>
#include <vector>
namespace base64 = boost::beast::detail::base64;

template <typename Out> Out load_file(std::string const& filename, Out out) {
    std::ifstream ifs(filename, std::ios::binary); // or "rb" on your fopen
    ifs.exceptions(std::ios::failbit |
                   std::ios::badbit); // we prefer exceptions
    return std::copy(std::istreambuf_iterator<char>(ifs), {}, out);
}

int main() {
    std::vector<char> input;
    load_file("filename.bin", back_inserter(input));

    // allocate "enough" space, using an upperbound prediction:
    std::string encoded(base64::encoded_size(input.size()), '[=13=]');

    // encode returns the **actual** encoded_size:
    auto encoded_size = base64::encode(encoded.data(), input.data(), input.size());
    encoded.resize(encoded_size); // so adjust the size

    std::ofstream("orig.bin", std::ios::binary)
        .write(input.data(), input.size());
    std::ofstream("encoded.txt") << encoded;

    // allocate "enough" space, using an upperbound prediction:
    std::vector<char> decoded(base64::decoded_size(encoded_size), 0);

    auto [decoded_size, // decode returns the **actual** decoded_size
          processed]    // (as well as number of encoded bytes processed)
        = base64::decode(decoded.data(), encoded.data(), encoded.size());
    decoded.resize(decoded_size); // so adjust the size

    std::ofstream("decoded.bin", std::ios::binary)
        .write(decoded.data(), decoded.size());
}

打印。当运行在“自身”上使用

g++ -std=c++20 -O2 -Wall -pedantic -pthread main.cpp -o filename.bin && ./filename.bin
md5sum filename.bin orig.bin decoded.bin
base64 -d < encoded.txt | md5sum

它打印

d4c96726eb621374fa1b7f0fa92025bf  filename.bin
d4c96726eb621374fa1b7f0fa92025bf  orig.bin
d4c96726eb621374fa1b7f0fa92025bf  decoded.bin
d4c96726eb621374fa1b7f0fa92025bf  -

base64 编码从 dos header 中删除回车 return

base64 encoding removing carriage return from dos header

c++

boost

base64

newline

carriage-return

结论：错在哪里？