`h5py` 中 `ros3` 驱动程序的自定义端点

Question

h5py

使用

ros3

驱动程序支持 HDF5 的本机 S3 驱动程序。我们已经通过

HDF5

.

的本地构建启用了此功能

>>> import h5py
>>>
>>> print(f'Registered drivers: {h5py.registered_drivers()}')

Registered drivers: frozenset({ 'ros3', 'sec2', 'fileobj', 'core', 'family', 'split', 'stdio', 'mpio'})

我们的 S3 服务（不是 AWS）有一个自定义端点。我们运行 Ceph/S3 服务。

有没有办法指定 S3 端点？

这里的文档没有提到它。

如果我们尝试运行以下命令，我们会收到一个一般错误，我们认为该错误与明显缺失的端点有关。

>>> h5py.File(
  f, 
  driver='ros3', 
  aws_region=bytes('unused', 'utf-8'),  # unused by our S3 but required
  secret_id=bytes(access_key, 'utf-8'), 
  secret_key=bytes(secret_key, 'utf-8')
)

Traceback (most recent call last):
  File "mycode.py", line 32, in <module>
    with h5py.File(f, driver='ros3', aws_region=b'west', secret_id=access_key, secret_key=secret_key) as fh5:
  File "opt/anaconda3/envs/smart_open_env/lib/python3.9/site-packages/h5py/_hl/files.py", line 567, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
  File "opt/anaconda3/envs/smart_open_env/lib/python3.9/site-packages/h5py/_hl/files.py", line 231, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 106, in h5py.h5f.open
OSError: Unable to open file (curl cannot perform request)