以 HDF5 格式从 1D 连续结构写入 3D 的 MPI 并行文件

Question

我正在尝试使用复合数据类型并行编写一组变量（dep_var class 的 gridinfo 对象）。我的系统总大小是 NX x NY x NZ，工作人员信息存储在工作人员（worker_info class 的对象）中。函数 "write_hdf(...)" 仅由工人执行。

现在，我的网格信息是一维对象数组，形式为 (i+offset_x) Ny*Nz+(k+offset_z)*Ny+(j+offset_y) 。我在末端有填充，由 offset_x、offset_y 和 offset_z 指定。当我希望我的数据空间成为 3D 数组时，我无法理解如何在内存空间中为 1D 数组指定此偏移量。请注意，main_offset_x,y,z 是工作块在数据空间上的偏移量。

使用 hdf 教程，我能够写到这里。如果您能为我提供一些有关如何前进的提示，我将不胜感激。提前致谢！！

void write_hdf(MPI_Comm comm,MPI_Func *mpi_obj,int iter){
    stringstream hdf_fname;
    hdf_fname<<"Out_"<<iter<<".h5";
    string hdf_filename = hdf_fname.str();
    //Create a new file collectively and release property list identifier.
    hid_t fapl_id=H5Pcreate(H5P_FILE_ACCESS);
    H5Pset_fapl_mpio(fapl_id,comm,MPI_INFO_NULL);
    hid_t file_id=H5Fcreate(hdf_filename.c_str(),H5F_ACC_TRUNC,H5P_DEFAULT,fapl_id);
    H5Pclose(fapl_id);

    int RANK=3;
    hsize_t dims[RANK];
    herr_t status;
    dims[0]=NX;dims[1]=NY;dims[2]=NZ;
    //Create dataspace
    hid_t dataspace_id=H5Screate_simple(RANK,dims,NULL);

    //Create compound datatype
    hid_t cmpd_dtype_id=H5Tcreate(H5T_COMPOUND,sizeof(dep_var));        
    status=H5Tinsert(cmpd_dtype_id, "Temperature", HOFFSET(dep_var,Temp),H5T_NATIVE_DOUBLE);
    status=H5Tinsert(cmpd_dtype_id, "fs", HOFFSET(dep_var,fs),H5T_NATIVE_FLOAT);
    status = H5Tset_order(cmpd_dtype_id, H5T_ORDER_LE);

    //Create chunk dataset
    dims[0]=worker->Nx;dims[1]=worker->Ny;dims[2]=worker->Nz;
    hid_t plist_id = H5Pcreate(H5P_DATASET_CREATE);
    H5Pset_chunk(plist_id, RANK, dims);
    hid_t dataset_id=H5Dcreate(file_id,"All_VAR",cmpd_dtype_id,dataspace_id,H5P_DEFAULT,plist_id,H5P_DEFAULT);
    H5Pclose(plist_id);
    H5Sclose(dataspace_id);

    //create memspace for each worker
    hsize_t offset[RANK],block[RANK],stride[RANK],count[RANK];
    hid_t memspace_id=H5Screate_simple(RANK,dims,NULL);
    count[0] =worker->Nx;count[1] = worker->Ny ;count[2]=worker->Nz;
    stride[0] = 1;stride[1] = 1;stride[2]=1;
    block[0] = dims[0];block[1] = dims[1];block[2]=dims[2];
    offset[0]=worker->main_offset_x;
    offset[1]=worker->main_offset_y;
    offset[2]=worker->main_offset_z;

    dataspace_id = H5Dget_space(dataset_id);
    status = H5Sselect_hyperslab(dataspace_id, H5S_SELECT_SET, offset, stride, count, block);

    //Create property list for collective dataset write.
    hid_t xf_id=H5Pcreate(H5P_DATASET_XFER);
    H5Pset_dxpl_mpio(xf_id,H5FD_MPIO_COLLECTIVE);

    //Write File
    status = H5Dwrite(dataset_id, cmpd_dtype_id, memspace_id, dataspace_id,xf_id, &(gridinfo[0]));

    //Free
    H5Sclose(dataspace_id);
    H5Sclose(memspace_id);
    H5Dclose(dataset_id);
    H5Pclose(xf_id);
    H5Fclose(file_id);
}

class dep_var{
    public:
        double Temp;
        double fs;
};

class worker_info{
    public:
        int Nx,Ny,Nz;//Worker size
        int real_Nx,real_Ny,real_Nz;//Size with padding
        int offset_x,offset_y,offset_z; //Offsets for padding
        int main_offset_x,main_offset_y,main_offset_z; //offsets for this worker on dataspace
};

enter image description here

Answer 1

It is all about indexing, we can regard 3D array as a "book" which has NZ pages with each page NX(column)*NY(row) words.  We now numbering words in this "book” row by row. 
Then the index of a word may be written as:
  index=NX*NY*z+y*NX+x;

此致。

以 HDF5 格式从 1D 连续结构写入 3D 的 MPI 并行文件

MPI Parallel File writing in HDF5 format from1D contiguous struct to 3D

parallel-processing

hdf5

mpich