c ++将向量写入hdf5文件而不存储在数组中

Question

我试图在 .h5 文件中存储 4 个向量（大小可变，但所有 4 个都具有相同的大小）。到目前为止，我是这样做的：

void write_file_h5(std::string filename,
            std::vector<int16_t> &x,
            std::vector<int16_t> &y,
            std::vector<int8_t> &p,
            std::vector<int32_t> &ts)
{
  struct myEvent myEvents[x.size()];
  for (int i=0; i<x.size();i++)
  {
    myEvents[i].x = x[i];
    myEvents[i].y = y[i];
    myEvents[i].p = int(p[i]);
    myEvents[i].ts = ts[i];
  }
  H5::CompType mtype(sizeof(myEvent));
  // Define the datatype to pass HDF5
  mtype.insertMember("x", HOFFSET(myEvent, x), H5::PredType::NATIVE_INT16);
  mtype.insertMember("y", HOFFSET(myEvent, y), H5::PredType::NATIVE_INT16);
  mtype.insertMember("p", HOFFSET(myEvent, p), H5::PredType::NATIVE_INT8);
  mtype.insertMember("ts", HOFFSET(myEvent, ts), H5::PredType::NATIVE_INT32);


  // preparation of a dataset and a file.
  hsize_t dim[1];
  dim[0] = sizeof(myEvents) / sizeof(myEvent);
  int rank = sizeof(dim) / sizeof(hsize_t);
  H5::DataSpace space(rank, dim);
  H5::H5File *file = new H5::H5File(filename, H5F_ACC_TRUNC);
  H5::DataSet *dataset = new H5::DataSet(file->createDataSet("myDataset", mtype, space));
  dataset->write(myEvents, mtype);
  delete dataset;
  delete file;
}

问题是：随着向量变大（向量大小 > 732428），我得到一个分段错误：

Segmentation fault (core dumped)

我想那是因为我要创建的数组太大了。有没有人知道如何解决这个问题？即如何在使用最少内存的同时编写向量？我需要向量，因为我需要在程序执行期间动态地填充它们...所以直接将值存储在数组中似乎是不可能的。

提前致谢

Answer 1

这一行

struct myEvent myEvents[x.size()];

创建一个 variable-length array，它是一个可能放置在堆栈上的非标准（在 C++ 中）数组。

将其替换为 std::vector<myEvent> 以在堆上分配内存。

void write_file_h5(std::string filename,
            std::vector<int16_t> &x,
            std::vector<int16_t> &y,
            std::vector<int8_t> &p,
            std::vector<int32_t> &ts)
{
  // Suggestion: check that y, p and ts has at least as many elements as x
  std::vector<myEvent> myEvents(x.size());  // replacement for the VLA

  for (int i=0; i<x.size();i++) {
    myEvents[i].x = x[i];
    myEvents[i].y = y[i];
    myEvents[i].p = int(p[i]);
    myEvents[i].ts = ts[i];
  }

  H5::CompType mtype(sizeof(myEvent));
  // Define the datatype to pass HDF5
  mtype.insertMember("x", HOFFSET(myEvent, x), H5::PredType::NATIVE_INT16);
  mtype.insertMember("y", HOFFSET(myEvent, y), H5::PredType::NATIVE_INT16);
  mtype.insertMember("p", HOFFSET(myEvent, p), H5::PredType::NATIVE_INT8);
  mtype.insertMember("ts", HOFFSET(myEvent, ts), H5::PredType::NATIVE_INT32);

  // preparation of a dataset and a file.
  hsize_t dim[1];
  dim[0] = myEvents.size();                   // using vector::size()
  int rank = sizeof(dim) / sizeof(hsize_t);
  H5::DataSpace space(rank, dim);

  H5::H5File *file = new H5::H5File(filename, H5F_ACC_TRUNC);
  H5::DataSet *dataset = new H5::DataSet(file->createDataSet("myDataset", mtype, space));
  dataset->write(myEvents.data(), mtype);     // use vector::data()
  delete dataset;
  delete file;
}

我还建议不要使用 new 和 delete 来简化函数的最后一部分：

  H5::H5File file(filename, H5F_ACC_TRUNC);
  H5::DataSet dataset(file.createDataSet("myDataset", mtype, space));
  dataset.write(myEvents.data(), mtype);

c ++将向量写入hdf5文件而不存储在数组中

c++ write vectors to hdf5-file without storing in array

c++

vector

hdf5