将分块数据数组写入HDF5

Question

我有这段代码可以使用 HDF5Sharp 将数组写入 Hdf5。 但问题是，我需要将数据写入 1 x 100 x 500 的块，而不是 100k x 100 x 500，而且我不知道该怎么做。

using HDF5CSharp;

// array of 100k x 100 x 500
float[,,] traindata = new float[100000, 100, 500];

// ...
// fill the array with data
// ...

long fileId = Hdf5.CreateFile("d:\\Temp\\Test.H5");
var groupId = Hdf5.CreateOrOpenGroup(fileId, "data");

Hdf5.WriteDataset(groupId, "data", traindata);

Hdf5.CloseGroup(groupId);
Hdf5.CloseFile(fileId);

Answer 1

不确定如何使用您指定的库完成此操作，但使用 HDFql（一种将您从 HDF5 的低级细节中抽象出来的高级语言），您的用例可以在 C# 中按以下方式解决：

// use HDFql namespace (make sure it can be found by the C# compiler)
using AS.HDFql;

// declare variables
float[,,] traindata = new float[1, 100, 500];
int number;
int i;

// create an HDF5 file named 'Test.h5' and use (i.e. open) it
HDFql.Execute("create and use file Test.h5");

// create a chunked dataset named 'data' (within a group named 'grp') of data type float with three dimensions (the first dimension is extendible)
HDFql.Execute("create chunked(1, 100, 500) dataset grp/data as float(0 to unlimited, 100, 500)");

// register variable 'traindata'
number = HDFql.VariableRegister(traindata);

// loop 100000 times
for(i = 0; i < 100000; i++)
{
   // populate variable 'traindata'
   // (...)

   // insert (i.e. write) data stored in 'traindata' into the last position of 'data' (using a hyperslab selection)
   HDFql.Execute("insert into grp/data[-1:::] values from memory " + number);

   // alter (i.e. extend) first dimension of 'data' plus one unit
   HDFql.Execute("alter dimension grp/data to +1");
}