使用带有二进制存档的 boost 序列化时出错
Error using boost serialization with binary archive
从 boost::archive::binary_iarchive 读入我的变量时出现以下错误：
test-serialization(9285,0x11c62fdc0) malloc: can't allocate region
*** mach_vm_map(size=18014398509486080) failed (error code=3)
test-serialization(9285,0x11c62fdc0) malloc: *** set a breakpoint in malloc_error_break to debug
我的序列化和反序列化代码是：
template<class Archive>
void save(Archive & archive, const helib::PubKey & pubkey, const unsigned int version){
BOOST_TEST_MESSAGE("inside save_construct_data");
archive << &(pubkey.context);
archive << pubkey.skBounds;
archive << pubkey.keySwitching;
archive << pubkey.keySwitchMap;
archive << pubkey.KS_strategy;
archive << pubkey.recryptKeyID;
}
template<class Archive>
void load_construct_data(Archive & archive, helib::PubKey * pubkey, const unsigned int version){
helib::Context * context = new helib::Context(2,3,1); //random numbers since there is no default constructor
BOOST_TEST_MESSAGE("deserializing context");
archive >> context;
std::vector<double> skBounds;
std::vector<helib::KeySwitch> keySwitching;
std::vector<std::vector<long>> keySwitchMap;
NTL::Vec<long> KS_strategy;
long recryptKeyID;
BOOST_TEST_MESSAGE("deserializing skbounds");
archive >> skBounds;
BOOST_TEST_MESSAGE("deserializing keyswitching");
archive >> keySwitching;
BOOST_TEST_MESSAGE("deserializing keyswitchmap");
archive >> keySwitchMap;
BOOST_TEST_MESSAGE("deserializing KS_strategy");
archive >> KS_strategy;
BOOST_TEST_MESSAGE("deserializing recryptKeyID");
archive >> recryptKeyID;
BOOST_TEST_MESSAGE("new pubkey");
::new(pubkey)helib::PubKey(*context);
//TODO: complete
}
template<class Archive>
void serialize(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
split_free(archive, pubkey, version);
}
template<class Archive>
void load(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
}
调用代码的测试如下：
BOOST_AUTO_TEST_CASE(serialization_pubkey)
{
auto context = helibTestContext();
helib::SecKey secret_key(context);
secret_key.GenSecKey();
// Compute key-switching matrices that we need
helib::addSome1DMatrices(secret_key);
// Set the secret key (upcast: SecKey is a subclass of PubKey)
const helib::PubKey& original_pubkey = secret_key;
std::string filename = "pubkey.serialized";
std::ofstream os(filename, std::ios::binary);
{
boost::archive::binary_oarchive oarchive(os);
oarchive << original_pubkey;
}
helib::PubKey * restored_pubkey = new helib::PubKey(helib::Context(2,3,1));
{
std::ifstream ifs(filename, std::ios::binary);
boost::archive::binary_iarchive iarchive(ifs);
BOOST_TEST_CHECKPOINT("calling deserialization");
iarchive >> restored_pubkey;
BOOST_TEST_CHECKPOINT("done with deserialization");
//tests ommitted
}
}
注意事项：
序列化在 boost::archive::text_oarchive 和 boost::archive::binary_oarchive 上都可以正常工作。他们分别创建了一个46M和21M的文件（大，我知道）。
反序列化 boost::archive::text_iarchive 基本上在执行 archive >> keySwitching; 时停止 进程自动终止。这实际上是存档的最大部分。
我决定尝试使用 boost::archive::binary_iarchive，因为文件大小只有一半，但我收到了开头显示的错误。执行第一次从存档读取时发生错误：archive >> context;.
输入和输出之间的不对称（save和load_construct_data）是因为我找不到另一种方法来避免执行派生的序列化class 的 helib::PubKey。使用指向 helib::PubKey 的指针给我编译错误，要求对派生的 class 进行序列化。如果有其他方法，我洗耳恭听。
感谢您的帮助。
更新:
我正在对加密库 HElib because I need to send ciphertext over the wire. One of these classes is helib::PubKey. I'm using the boost serialization library for the implementation. I have created a gist 中的某些 classes 实施反序列化，以提供评论中建议的 reprex。有 3 个文件：
- serialization.hpp，它包含序列化实现。不幸的是，
helib::PubKey 依赖于许多其他 classes 使文件相当长。所有其他 classes 都有通过的单元测试。 此外，为了序列化它，我不得不对 class 进行微小的修改。我将 public 设置为私人成员 。
- test-serialization.cpp，包含单元测试。
- 生成文件。 运行 make 创建可执行文件 test-serialization。
vector<bool> 再次出击
它实际上在我的测试盒上分配了 0x1fffffffff20000 位（即 144 petabits）。这直接来自 IndexSet::resize().
现在我对这里使用 std::vector<bool> 的 HElib 有严重的疑问（似乎 far 更好地服务于 boost::icl::interval_set<> 之类的东西）。
嗯。那是一场徒劳的追逐（IndexSet 序列化可以得到很大改进）。但是，真正的问题是您有 Undefined Behaviour 因为您没有反序列化与序列化相同的类型。
您序列化了 PubKey，但尝试反序列化为 PubKey*。呃
除此之外，还有很多问题：
您必须修改库以创建私有成员 public。这很容易违反 ODR（使 class 布局不兼容）。
您似乎将上下文视为 "dynamic" 资源，这将吸引 Object Tracking。这可能是一个可行的方法。但。你必须考虑所有权。
看来你还没有这样做。例如，load_construct_data 中 DoublCRT 的行是明确的内存泄漏：
helib::Context * context = new helib::Context(2,3,1);
您永远不会使用它，也永远不会释放它。事实上，您只需用反序列化的实例覆盖它，可能拥有也可能不拥有。第二十二条军规
load_construct_data 中 PubKey 的情况完全相同。
更糟糕的是，在 save_construct_data 中，您完全无偿地为每个 DoubleCRT 中的每个 SecKey:
复制上下文对象
auto context = polynomial->getContext();
archive << &context;
因为你伪造它作为指针序列化，再次（显然无用）对象跟踪开始，只是意味着你序列化了冗余的Context副本，这将是所有在反序列化时被泄露。
我很想假设两者中的上下文实例总是相同的？为什么不单独序列化上下文呢？
事实上，我去分析了 HElib 源代码以检查这些假设。事实证明我是对的。没有什么能在外部构建上下文
std::unique_ptr<Context> buildContextFromBinary(std::istream& str);
std::unique_ptr<Context> buildContextFromAscii(std::istream& str);
如您所见，他们 return 拥有指针。你应该一直在使用它们。也许即使有内置的序列化，我在这里也几乎被绊倒了。
重组时间
我会使用 HElib 的序列化代码（因为，为什么要重新发明轮子并这样做会产生大量错误？）。如果您坚持 与 Boost Serialization 集成，您就可以吃蛋糕了：
template <class Archive> void save(Archive& archive, const helib::PubKey& pubkey, unsigned) {
using V = std::vector<char>;
using D = iostreams::back_insert_device<V>;
V data;
{
D dev(data);
iostreams::stream_buffer<D> sbuf(dev);
std::ostream os(&sbuf); // expose as std::ostream
helib::writePubKeyBinary(os, pubkey);
}
archive << data;
}
template <class Archive> void load(Archive& archive, helib::PubKey& pubkey, unsigned) {
std::vector<char> data;
archive >> data;
using S = iostreams::array_source;
S source(data.data(), data.size());
iostreams::stream_buffer<S> sbuf(source);
{
std::istream is(&sbuf); // expose as std::istream
helib::readPubKeyBinary(is, pubkey);
}
}
就是这样。 24行代码。它将由图书馆作者进行测试和维护。你无法击败它（显然）。我对测试做了一些修改，这样我们就不会再滥用私人信息了。
清理代码
通过分离出一个 helper 来处理 blob 写入，我们可以以非常相似的方式实现不同的 helib 类型：
namespace helib { // leverage ADL
template <class A> void save(A& ar, const Context& o, unsigned) {
Blob data = to_blob(o, writeContextBinary);
ar << data;
}
template <class A> void load(A& ar, Context& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readContextBinary);
}
template <class A> void save(A& ar, const PubKey& o, unsigned) {
Blob data = to_blob(o, writePubKeyBinary);
ar << data;
}
template <class A> void load(A& ar, PubKey& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readPubKeyBinary);
}
}
这对我来说很优雅。
完整列表
我克隆了一个新的要点 https://gist.github.com/sehe/ba82a0329e4ec586363eb82d3f3b9326，其中包括以下变更集：
0079c07 Make it compile locally
b3b2cf1 Squelch the warnings
011b589 Endof investigations, regroup time
f4d79a6 Reimplemented using HElib binary IO
a403e97 Bitwise reproducible outputs
只有最后两个提交包含与实际修复相关的更改。
我也会在这里列出完整的代码以供后代使用。测试代码中有许多微妙的重组和同上注释。您最好仔细阅读它们，看看您是否理解它们以及它们的含义是否适合您的需要。我留下了评论，描述了为什么测试断言是有用的。
文件serialization.hpp
#ifndef EVOTING_SERIALIZATION_H
#define EVOTING_SERIALIZATION_H
#define BOOST_TEST_MODULE main
#include <helib/helib.h>
#include <boost/serialization/split_free.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/iostreams/stream_buffer.hpp>
#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/device/array.hpp>
namespace /* file-static */ {
using Blob = std::vector<char>;
template <typename T, typename F>
Blob to_blob(const T& object, F writer) {
using D = boost::iostreams::back_insert_device<Blob>;
Blob data;
{
D dev(data);
boost::iostreams::stream_buffer<D> sbuf(dev);
std::ostream os(&sbuf); // expose as std::ostream
writer(os, object);
}
return data;
}
template <typename T, typename F>
void from_blob(Blob const& data, T& object, F reader) {
boost::iostreams::stream_buffer<boost::iostreams::array_source>
sbuf(data.data(), data.size());
std::istream is(&sbuf); // expose as std::istream
reader(is, object);
}
}
namespace helib { // leverage ADL
template <class A> void save(A& ar, const Context& o, unsigned) {
Blob data = to_blob(o, writeContextBinary);
ar << data;
}
template <class A> void load(A& ar, Context& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readContextBinary);
}
template <class A> void save(A& ar, const PubKey& o, unsigned) {
Blob data = to_blob(o, writePubKeyBinary);
ar << data;
}
template <class A> void load(A& ar, PubKey& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readPubKeyBinary);
}
}
BOOST_SERIALIZATION_SPLIT_FREE(helib::Context)
BOOST_SERIALIZATION_SPLIT_FREE(helib::PubKey)
#endif //EVOTING_SERIALIZATION_H
文件test-serialization.cpp
#define BOOST_TEST_MODULE main
#include <boost/test/included/unit_test.hpp>
#include <helib/helib.h>
#include <fstream>
#include "serialization.hpp"
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
helib::Context helibTestMinimalContext(){
// Plaintext prime modulus
unsigned long p = 4999;
// Cyclotomic polynomial - defines phi(m)
unsigned long m = 32109;
// Hensel lifting (default = 1)
unsigned long r = 1;
return helib::Context(m, p, r);
}
helib::Context helibTestContext(){
auto context = helibTestMinimalContext();
// Number of bits of the modulus chain
unsigned long bits = 300;
// Number of columns of Key-Switching matix (default = 2 or 3)
unsigned long c = 2;
// Modify the context, adding primes to the modulus chain
buildModChain(context, bits, c);
return context;
}
BOOST_AUTO_TEST_CASE(serialization_pubkey) {
auto context = helibTestContext();
helib::SecKey secret_key(context);
secret_key.GenSecKey();
// Compute key-switching matrices that we need
helib::addSome1DMatrices(secret_key);
// Set the secret key (upcast: SecKey is a subclass of PubKey)
const helib::PubKey& original_pubkey = secret_key;
std::string const filename = "pubkey.serialized";
{
std::ofstream os(filename, std::ios::binary);
boost::archive::binary_oarchive oarchive(os);
oarchive << context << original_pubkey;
}
{
// just checking reproducible output
std::ofstream os(filename + ".2", std::ios::binary);
boost::archive::binary_oarchive oarchive(os);
oarchive << context << original_pubkey;
}
// reading back to independent instances of Context/PubKey
{
// NOTE: if you start from something rogue, it will fail with PAlgebra mismatch.
helib::Context surrogate = helibTestMinimalContext();
std::ifstream ifs(filename, std::ios::binary);
boost::archive::binary_iarchive iarchive(ifs);
iarchive >> surrogate;
// we CAN test that the contexts end up matching
BOOST_TEST((context == surrogate));
helib::SecKey independent(surrogate);
helib::PubKey& indep_pk = independent;
iarchive >> indep_pk;
// private again, as it should be, but to understand the relation:
// BOOST_TEST((&independent.context == &surrogate));
// The library's operator== compares the reference, so it would say "not equal"
BOOST_TEST((indep_pk != original_pubkey));
{
// just checking reproducible output
std::ofstream os(filename + ".3", std::ios::binary);
boost::archive::binary_oarchive oarchive(os);
oarchive << surrogate << indep_pk;
}
}
// doing it the other way (sharing the context):
{
helib::PubKey restored_pubkey(context);
{
std::ifstream ifs(filename, std::ios::binary);
boost::archive::binary_iarchive iarchive(ifs);
iarchive >> context >> restored_pubkey;
}
// now `operator==` confirms equality
BOOST_TEST((restored_pubkey == original_pubkey));
{
// just checking reproducible output
std::ofstream os(filename + ".4", std::ios::binary);
boost::archive::binary_oarchive oarchive(os);
oarchive << context << restored_pubkey;
}
}
}
测试输出
time ./test-serialization -l all -r detailed
Running 1 test case...
Entering test module "main"
test-serialization.cpp(34): Entering test case "serialization_pubkey"
test-serialization.cpp(61): info: check (context == surrogate) has passed
test-serialization.cpp(70): info: check (indep_pk != original_pubkey) has passed
test-serialization.cpp(82): info: check (restored_pubkey == original_pubkey) has passed
test-serialization.cpp(34): Leaving test case "serialization_pubkey"; testing time: 36385217us
Leaving test module "main"; testing time: 36385273us
Test module "main" has passed with:
1 test case out of 1 passed
3 assertions out of 3 passed
Test case "serialization_pubkey" has passed with:
3 assertions out of 3 passed
real 0m36,698s
user 0m35,558s
sys 0m0,850s
按位可重现输出
在重复序列化时，输出确实按位相同，这可能是一个重要的 属性:
sha256sum pubkey.serialized*
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.2
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.3
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.4
请注意，它（显然）在运行中并不相同（因为它生成不同的密钥 material）。
支线任务（追雁）
手动改进 IndexSet 序列化代码的一种方法是也使用 vector<bool>:
template<class Archive>
void save(Archive & archive, const helib::IndexSet & index_set, const unsigned int version){
std::vector<bool> elements;
elements.resize(index_set.last()-index_set.first()+1);
for (auto n : index_set)
elements[n-index_set.first()] = true;
archive << index_set.first() << elements;
}
template<class Archive>
void load(Archive & archive, helib::IndexSet & index_set, const unsigned int version){
long first_ = 0;
std::vector<bool> elements;
archive >> first_ >> elements;
index_set.clear();
for (size_t n = 0; n < elements.size(); ++n) {
if (elements[n])
index_set.insert(n+first_);
}
}
更好的办法是使用 dynamic_bitset（为此我恰好有 contributed the serialization code (see ））：
template<class Archive>
void save(Archive & archive, const helib::IndexSet & index_set, const unsigned int version){
boost::dynamic_bitset<> elements;
elements.resize(index_set.last()-index_set.first()+1);
for (auto n : index_set)
elements.set(n-index_set.first());
archive << index_set.first() << elements;
}
template<class Archive>
void load(Archive & archive, helib::IndexSet & index_set, const unsigned int version) {
long first_ = 0;
boost::dynamic_bitset<> elements;
archive >> first_ >> elements;
index_set.clear();
for (size_t n = elements.find_first(); n != -1; n = elements.find_next(n))
index_set.insert(n+first_);
}
Of course, you would likely have to do similar things for IndexMap.
从 boost::archive::binary_iarchive 读入我的变量时出现以下错误：
test-serialization(9285,0x11c62fdc0) malloc: can't allocate region
*** mach_vm_map(size=18014398509486080) failed (error code=3)
test-serialization(9285,0x11c62fdc0) malloc: *** set a breakpoint in malloc_error_break to debug
我的序列化和反序列化代码是：
template<class Archive>
void save(Archive & archive, const helib::PubKey & pubkey, const unsigned int version){
BOOST_TEST_MESSAGE("inside save_construct_data");
archive << &(pubkey.context);
archive << pubkey.skBounds;
archive << pubkey.keySwitching;
archive << pubkey.keySwitchMap;
archive << pubkey.KS_strategy;
archive << pubkey.recryptKeyID;
}
template<class Archive>
void load_construct_data(Archive & archive, helib::PubKey * pubkey, const unsigned int version){
helib::Context * context = new helib::Context(2,3,1); //random numbers since there is no default constructor
BOOST_TEST_MESSAGE("deserializing context");
archive >> context;
std::vector<double> skBounds;
std::vector<helib::KeySwitch> keySwitching;
std::vector<std::vector<long>> keySwitchMap;
NTL::Vec<long> KS_strategy;
long recryptKeyID;
BOOST_TEST_MESSAGE("deserializing skbounds");
archive >> skBounds;
BOOST_TEST_MESSAGE("deserializing keyswitching");
archive >> keySwitching;
BOOST_TEST_MESSAGE("deserializing keyswitchmap");
archive >> keySwitchMap;
BOOST_TEST_MESSAGE("deserializing KS_strategy");
archive >> KS_strategy;
BOOST_TEST_MESSAGE("deserializing recryptKeyID");
archive >> recryptKeyID;
BOOST_TEST_MESSAGE("new pubkey");
::new(pubkey)helib::PubKey(*context);
//TODO: complete
}
template<class Archive>
void serialize(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
split_free(archive, pubkey, version);
}
template<class Archive>
void load(Archive & archive, helib::PubKey & pubkey, const unsigned int version){
}
调用代码的测试如下：
BOOST_AUTO_TEST_CASE(serialization_pubkey)
{
auto context = helibTestContext();
helib::SecKey secret_key(context);
secret_key.GenSecKey();
// Compute key-switching matrices that we need
helib::addSome1DMatrices(secret_key);
// Set the secret key (upcast: SecKey is a subclass of PubKey)
const helib::PubKey& original_pubkey = secret_key;
std::string filename = "pubkey.serialized";
std::ofstream os(filename, std::ios::binary);
{
boost::archive::binary_oarchive oarchive(os);
oarchive << original_pubkey;
}
helib::PubKey * restored_pubkey = new helib::PubKey(helib::Context(2,3,1));
{
std::ifstream ifs(filename, std::ios::binary);
boost::archive::binary_iarchive iarchive(ifs);
BOOST_TEST_CHECKPOINT("calling deserialization");
iarchive >> restored_pubkey;
BOOST_TEST_CHECKPOINT("done with deserialization");
//tests ommitted
}
}
注意事项：
序列化在
boost::archive::text_oarchive和boost::archive::binary_oarchive上都可以正常工作。他们分别创建了一个46M和21M的文件（大，我知道）。反序列化
boost::archive::text_iarchive基本上在执行archive >> keySwitching;时停止 进程自动终止。这实际上是存档的最大部分。我决定尝试使用
boost::archive::binary_iarchive，因为文件大小只有一半，但我收到了开头显示的错误。执行第一次从存档读取时发生错误：archive >> context;.输入和输出之间的不对称（
save和load_construct_data）是因为我找不到另一种方法来避免执行派生的序列化class 的helib::PubKey。使用指向helib::PubKey的指针给我编译错误，要求对派生的 class 进行序列化。如果有其他方法，我洗耳恭听。
感谢您的帮助。
更新:
我正在对加密库 HElib because I need to send ciphertext over the wire. One of these classes is helib::PubKey. I'm using the boost serialization library for the implementation. I have created a gist 中的某些 classes 实施反序列化，以提供评论中建议的 reprex。有 3 个文件：
- serialization.hpp，它包含序列化实现。不幸的是，
helib::PubKey依赖于许多其他 classes 使文件相当长。所有其他 classes 都有通过的单元测试。 此外，为了序列化它，我不得不对 class 进行微小的修改。我将 public 设置为私人成员 。 - test-serialization.cpp，包含单元测试。
- 生成文件。 运行 make 创建可执行文件 test-serialization。
vector<bool> 再次出击
它实际上在我的测试盒上分配了 0x1fffffffff20000 位（即 144 petabits）。这直接来自 IndexSet::resize().
现在我对这里使用 std::vector<bool> 的 HElib 有严重的疑问（似乎 far 更好地服务于 boost::icl::interval_set<> 之类的东西）。
嗯。那是一场徒劳的追逐（IndexSet 序列化可以得到很大改进）。但是，真正的问题是您有 Undefined Behaviour 因为您没有反序列化与序列化相同的类型。
您序列化了 PubKey，但尝试反序列化为 PubKey*。呃
除此之外，还有很多问题：
您必须修改库以创建私有成员 public。这很容易违反 ODR（使 class 布局不兼容）。
您似乎将上下文视为 "dynamic" 资源，这将吸引 Object Tracking。这可能是一个可行的方法。但。你必须考虑所有权。
看来你还没有这样做。例如，
load_construct_data中DoublCRT的行是明确的内存泄漏：helib::Context * context = new helib::Context(2,3,1);您永远不会使用它，也永远不会释放它。事实上，您只需用反序列化的实例覆盖它，可能拥有也可能不拥有。第二十二条军规
load_construct_data中PubKey的情况完全相同。更糟糕的是，在
复制上下文对象save_construct_data中，您完全无偿地为每个DoubleCRT中的每个SecKey:auto context = polynomial->getContext(); archive << &context;因为你伪造它作为指针序列化，再次（显然无用）对象跟踪开始，只是意味着你序列化了冗余的
Context副本，这将是所有在反序列化时被泄露。我很想假设两者中的上下文实例总是相同的？为什么不单独序列化上下文呢？
事实上，我去分析了 HElib 源代码以检查这些假设。事实证明我是对的。没有什么能在外部构建上下文
std::unique_ptr<Context> buildContextFromBinary(std::istream& str); std::unique_ptr<Context> buildContextFromAscii(std::istream& str);如您所见，他们 return 拥有指针。你应该一直在使用它们。也许即使有内置的序列化，我在这里也几乎被绊倒了。
重组时间
我会使用 HElib 的序列化代码（因为，为什么要重新发明轮子并这样做会产生大量错误？）。如果您坚持 与 Boost Serialization 集成，您就可以吃蛋糕了：
template <class Archive> void save(Archive& archive, const helib::PubKey& pubkey, unsigned) {
using V = std::vector<char>;
using D = iostreams::back_insert_device<V>;
V data;
{
D dev(data);
iostreams::stream_buffer<D> sbuf(dev);
std::ostream os(&sbuf); // expose as std::ostream
helib::writePubKeyBinary(os, pubkey);
}
archive << data;
}
template <class Archive> void load(Archive& archive, helib::PubKey& pubkey, unsigned) {
std::vector<char> data;
archive >> data;
using S = iostreams::array_source;
S source(data.data(), data.size());
iostreams::stream_buffer<S> sbuf(source);
{
std::istream is(&sbuf); // expose as std::istream
helib::readPubKeyBinary(is, pubkey);
}
}
就是这样。 24行代码。它将由图书馆作者进行测试和维护。你无法击败它（显然）。我对测试做了一些修改，这样我们就不会再滥用私人信息了。
清理代码
通过分离出一个 helper 来处理 blob 写入，我们可以以非常相似的方式实现不同的 helib 类型：
namespace helib { // leverage ADL
template <class A> void save(A& ar, const Context& o, unsigned) {
Blob data = to_blob(o, writeContextBinary);
ar << data;
}
template <class A> void load(A& ar, Context& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readContextBinary);
}
template <class A> void save(A& ar, const PubKey& o, unsigned) {
Blob data = to_blob(o, writePubKeyBinary);
ar << data;
}
template <class A> void load(A& ar, PubKey& o, unsigned) {
Blob data;
ar >> data;
from_blob(data, o, readPubKeyBinary);
}
}
这对我来说很优雅。
完整列表
我克隆了一个新的要点 https://gist.github.com/sehe/ba82a0329e4ec586363eb82d3f3b9326，其中包括以下变更集：
0079c07 Make it compile locally
b3b2cf1 Squelch the warnings
011b589 Endof investigations, regroup time
f4d79a6 Reimplemented using HElib binary IO
a403e97 Bitwise reproducible outputs
只有最后两个提交包含与实际修复相关的更改。
我也会在这里列出完整的代码以供后代使用。测试代码中有许多微妙的重组和同上注释。您最好仔细阅读它们，看看您是否理解它们以及它们的含义是否适合您的需要。我留下了评论，描述了为什么测试断言是有用的。
文件
serialization.hpp#ifndef EVOTING_SERIALIZATION_H #define EVOTING_SERIALIZATION_H #define BOOST_TEST_MODULE main #include <helib/helib.h> #include <boost/serialization/split_free.hpp> #include <boost/serialization/vector.hpp> #include <boost/iostreams/stream_buffer.hpp> #include <boost/iostreams/device/back_inserter.hpp> #include <boost/iostreams/device/array.hpp> namespace /* file-static */ { using Blob = std::vector<char>; template <typename T, typename F> Blob to_blob(const T& object, F writer) { using D = boost::iostreams::back_insert_device<Blob>; Blob data; { D dev(data); boost::iostreams::stream_buffer<D> sbuf(dev); std::ostream os(&sbuf); // expose as std::ostream writer(os, object); } return data; } template <typename T, typename F> void from_blob(Blob const& data, T& object, F reader) { boost::iostreams::stream_buffer<boost::iostreams::array_source> sbuf(data.data(), data.size()); std::istream is(&sbuf); // expose as std::istream reader(is, object); } } namespace helib { // leverage ADL template <class A> void save(A& ar, const Context& o, unsigned) { Blob data = to_blob(o, writeContextBinary); ar << data; } template <class A> void load(A& ar, Context& o, unsigned) { Blob data; ar >> data; from_blob(data, o, readContextBinary); } template <class A> void save(A& ar, const PubKey& o, unsigned) { Blob data = to_blob(o, writePubKeyBinary); ar << data; } template <class A> void load(A& ar, PubKey& o, unsigned) { Blob data; ar >> data; from_blob(data, o, readPubKeyBinary); } } BOOST_SERIALIZATION_SPLIT_FREE(helib::Context) BOOST_SERIALIZATION_SPLIT_FREE(helib::PubKey) #endif //EVOTING_SERIALIZATION_H文件
test-serialization.cpp#define BOOST_TEST_MODULE main #include <boost/test/included/unit_test.hpp> #include <helib/helib.h> #include <fstream> #include "serialization.hpp" #include <boost/archive/text_oarchive.hpp> #include <boost/archive/text_iarchive.hpp> #include <boost/archive/binary_oarchive.hpp> #include <boost/archive/binary_iarchive.hpp> helib::Context helibTestMinimalContext(){ // Plaintext prime modulus unsigned long p = 4999; // Cyclotomic polynomial - defines phi(m) unsigned long m = 32109; // Hensel lifting (default = 1) unsigned long r = 1; return helib::Context(m, p, r); } helib::Context helibTestContext(){ auto context = helibTestMinimalContext(); // Number of bits of the modulus chain unsigned long bits = 300; // Number of columns of Key-Switching matix (default = 2 or 3) unsigned long c = 2; // Modify the context, adding primes to the modulus chain buildModChain(context, bits, c); return context; } BOOST_AUTO_TEST_CASE(serialization_pubkey) { auto context = helibTestContext(); helib::SecKey secret_key(context); secret_key.GenSecKey(); // Compute key-switching matrices that we need helib::addSome1DMatrices(secret_key); // Set the secret key (upcast: SecKey is a subclass of PubKey) const helib::PubKey& original_pubkey = secret_key; std::string const filename = "pubkey.serialized"; { std::ofstream os(filename, std::ios::binary); boost::archive::binary_oarchive oarchive(os); oarchive << context << original_pubkey; } { // just checking reproducible output std::ofstream os(filename + ".2", std::ios::binary); boost::archive::binary_oarchive oarchive(os); oarchive << context << original_pubkey; } // reading back to independent instances of Context/PubKey { // NOTE: if you start from something rogue, it will fail with PAlgebra mismatch. helib::Context surrogate = helibTestMinimalContext(); std::ifstream ifs(filename, std::ios::binary); boost::archive::binary_iarchive iarchive(ifs); iarchive >> surrogate; // we CAN test that the contexts end up matching BOOST_TEST((context == surrogate)); helib::SecKey independent(surrogate); helib::PubKey& indep_pk = independent; iarchive >> indep_pk; // private again, as it should be, but to understand the relation: // BOOST_TEST((&independent.context == &surrogate)); // The library's operator== compares the reference, so it would say "not equal" BOOST_TEST((indep_pk != original_pubkey)); { // just checking reproducible output std::ofstream os(filename + ".3", std::ios::binary); boost::archive::binary_oarchive oarchive(os); oarchive << surrogate << indep_pk; } } // doing it the other way (sharing the context): { helib::PubKey restored_pubkey(context); { std::ifstream ifs(filename, std::ios::binary); boost::archive::binary_iarchive iarchive(ifs); iarchive >> context >> restored_pubkey; } // now `operator==` confirms equality BOOST_TEST((restored_pubkey == original_pubkey)); { // just checking reproducible output std::ofstream os(filename + ".4", std::ios::binary); boost::archive::binary_oarchive oarchive(os); oarchive << context << restored_pubkey; } } }
测试输出
time ./test-serialization -l all -r detailed
Running 1 test case...
Entering test module "main"
test-serialization.cpp(34): Entering test case "serialization_pubkey"
test-serialization.cpp(61): info: check (context == surrogate) has passed
test-serialization.cpp(70): info: check (indep_pk != original_pubkey) has passed
test-serialization.cpp(82): info: check (restored_pubkey == original_pubkey) has passed
test-serialization.cpp(34): Leaving test case "serialization_pubkey"; testing time: 36385217us
Leaving test module "main"; testing time: 36385273us
Test module "main" has passed with:
1 test case out of 1 passed
3 assertions out of 3 passed
Test case "serialization_pubkey" has passed with:
3 assertions out of 3 passed
real 0m36,698s
user 0m35,558s
sys 0m0,850s
按位可重现输出
在重复序列化时，输出确实按位相同，这可能是一个重要的 属性:
sha256sum pubkey.serialized*
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.2
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.3
66b95adbd996b100bff58774e066e7a309e70dff7cbbe08b5c77b9fa0f63c97f pubkey.serialized.4
请注意，它（显然）在运行中并不相同（因为它生成不同的密钥 material）。
支线任务（追雁）
手动改进 IndexSet 序列化代码的一种方法是也使用 vector<bool>:
template<class Archive>
void save(Archive & archive, const helib::IndexSet & index_set, const unsigned int version){
std::vector<bool> elements;
elements.resize(index_set.last()-index_set.first()+1);
for (auto n : index_set)
elements[n-index_set.first()] = true;
archive << index_set.first() << elements;
}
template<class Archive>
void load(Archive & archive, helib::IndexSet & index_set, const unsigned int version){
long first_ = 0;
std::vector<bool> elements;
archive >> first_ >> elements;
index_set.clear();
for (size_t n = 0; n < elements.size(); ++n) {
if (elements[n])
index_set.insert(n+first_);
}
}
更好的办法是使用 dynamic_bitset（为此我恰好有 contributed the serialization code (see
template<class Archive>
void save(Archive & archive, const helib::IndexSet & index_set, const unsigned int version){
boost::dynamic_bitset<> elements;
elements.resize(index_set.last()-index_set.first()+1);
for (auto n : index_set)
elements.set(n-index_set.first());
archive << index_set.first() << elements;
}
template<class Archive>
void load(Archive & archive, helib::IndexSet & index_set, const unsigned int version) {
long first_ = 0;
boost::dynamic_bitset<> elements;
archive >> first_ >> elements;
index_set.clear();
for (size_t n = elements.find_first(); n != -1; n = elements.find_next(n))
index_set.insert(n+first_);
}
Of course, you would likely have to do similar things for
IndexMap.