HDFql顺序写入结构数据集

HDFql sequential writing to the structure dataset

我正在尝试用 hdf5 编写一个简单的结构 但是我仍然无法将记录添加到已经创建的数据集中。 任务本身是存储一组任意长度的字节及其大小

void WriteStructToFile(vector<vector<char>> fl_stream)
{
    struct data
    {
        int size_array;
        char * value_array;
    };
    int status;
    string  query;
    string FileName = "Fly_stream_file.h5";
    string NameGroup = "Stream";
    string Namedataset = "Inclusions";

 //------------------------------- Created File and Dataset -------------
    query = "CREATE TRUNCATE FILE " + FileName;
    status = HDFql::execute(query.c_str());
    query = "USE FILE " + FileName;
    status = HDFql::execute(query.c_str());
    query = "CREATE GROUP " + NameGroup;
    status = HDFql::execute(query.c_str());
    query = "CREATE TRUNCATE CHUNKED DATASET "+ Namedataset +" AS COMPOUND(size_array AS INT, value_array AS VARCHAR)(UNLIMITED)";
    stringstream scriptst;
    scriptst <<"CREATE TRUNCATE CHUNKED DATASET "<< Namedataset <<" AS COMPOUND("<< "size_array AS INT OFFSET " << offsetof(struct data, size_array)<<" ,"<<" value_array AS VARCHAR OFFSET "<< offsetof(struct data, value_array) <<")(UNLIMITED)"<< " SIZE "<< sizeof(struct data);
    status = HDFql::execute(scriptst);
    //clear request
    scriptst.str(std::string());
    scriptst.clear();

//--------------- Fill Dataset ----------------
    //we simulate the arrival of data in a function 
    
    
    struct data realdata;
    int number = HDFql::variableRegister(&realdata);
    for (int i = 0;i < fl_stream.size();i++)
    {

        realdata.size_array = fl_stream[i].size();
        realdata.value_array = fl_stream[i].data();

        //write data to the dataset
        scriptst << "INSERT INTO " << Namedataset << "(-1) VALUES FROM MEMORY " << number<< " SIZE " << sizeof(struct data) << " OFFSET(" << offsetof(struct data, size_array) << ", "
<< offsetof(struct data,value_array) << ")";
        status = HDFql::execute(scriptst);

        HDFql::execute("ALTER DIMENSION Inclusions TO +1");
        scriptst.str(std::string());
        scriptst.clear();
    }
    HDFql::variableUnregister(&realdata);
status = HDFql::execute("CLOSE FILE");
}

the result of the corrected code

创建数据集的脚本存在语法错误,缺少成员偏移量和化合物大小的规范。应该是这样的:

sprintf(script, "CREATE TRUNCATE CHUNKED DATASET Inclusions AS COMPOUND(sizear AS INT OFFSET %d, value AS VARCHAR OFFSET %d)(UNLIMITED) SIZE %d", offsetof(struct data, sizear), offsetof(struct data, value), sizeof(struct data));
status = HDFql::execute(script);

此外,您对数据集 Inclusions 进行的第二次写入应使用 hyperslab/point 选择(否则,之前写入的数据将被覆盖)。因此,请执行以下操作:

sprintf(script, "INSERT INTO Inclusions(-1) VALUES FROM MEMORY %d SIZE %d OFFSET(%d, %d)", number, sizeof(struct data), offsetof(struct data, sizear), offsetof(struct data, value));
hdfql_execute(script);

此外,将struct data的成员value的数据类型改为char *value;

最后,为了简化代码,从字符串 query 中删除对函数 c_str() 的调用,因为您可以将它直接传递给函数 HDFql::execute().