将分块数据数组写入 HDF5

Question

我有这段代码可以使用 HDF5Sharp 将数组写入 Hdf5。但问题是我需要将数据写入 1 x 100 x 500 而不是 100k x 100 x 500 的块中，我不知道该怎么做。

using HDF5CSharp;

// array of 100k x 100 x 500
float[,,] traindata = new float[100000, 100, 500];

// ...
// fill the array with data
// ...

long fileId = Hdf5.CreateFile("d:\Temp\Test.H5");
var groupId = Hdf5.CreateOrOpenGroup(fileId, "data");

Hdf5.WriteDataset(groupId, "data", traindata);

Hdf5.CloseGroup(groupId);
Hdf5.CloseFile(fileId);

Answer 1

不确定如何使用您指定的库完成此操作，但使用 HDFql，一种 high-level 语言，将您从 HDF5 的 low-level 细节中抽象出来，您的 use-case在C#中可以这样解决：

// use HDFql namespace (make sure it can be found by the C# compiler)
using AS.HDFql;

// declare variables
float[,,] traindata = new float[1, 100, 500];
int number;
int i;

// create an HDF5 file named 'Test.h5' and use (i.e. open) it
HDFql.Execute("create and use file Test.h5");

// create a chunked dataset named 'data' (within a group named 'grp') of data type float with three dimensions (the first dimension is extendible)
HDFql.Execute("create chunked(1, 100, 500) dataset grp/data as float(0 to unlimited, 100, 500)");

// register variable 'traindata'
number = HDFql.VariableRegister(traindata);

// loop 100000 times
for(i = 0; i < 100000; i++)
{
   // populate variable 'traindata'
   // (...)

   // insert (i.e. write) data stored in 'traindata' into the last position of 'data' (using a hyperslab selection)
   HDFql.Execute("insert into grp/data(-1:::) values from memory " + number);

   // alter (i.e. extend) first dimension of 'data' plus one unit
   HDFql.Execute("alter dimension grp/data to +1");
}

将分块数据数组写入 HDF5

Write Chunked data array to HDF5

c#

arrays

multidimensional-array

hdf5