Copyrighrt (c) 2014-2020 UPVI, LLC
Home  Products  Live HDF5  Details

Live HDF5 - Details

Philosophy and Technology

This toolkit attempts to make it as easy as possible to use the HDF5 library. It follows the standard paradigm of "open, read/write, close." Open a file, write a dataset, read a dataset, close the file. The magic happens during the reads and writes. During writes, the library will attempt to map the LabVIEW data type to a corresponding HDF5 data type. Integers -> integers, floats -> floats, clusters -> compounds, etc. If the library is unable to handle a particular data type, the write node will not allow you to wire the data type to the node--the wire will be broken. (If using the variant version of the node, a run-time error will occur.) The library is able to handle arbitrarily complex data types, including clusters, strings, clusters of clusters of strings, etc.

LVHDF5 automatically maps most LabVIEW types to HDF5 and most HDF5 types to LabVIEW.
(I don't recommend that you use this particular datatype, but you could!)

During read operations, much like a LabVIEW primitive node, the dataset and attribute nodes take a datatype input. If the datatype input is wired, the toolkit will attempt to cast the data in the HDF5 file to the datatype wired to the node. If it is unable to do so, a runtime error (or possibly crash) will occur. Typically, the same datatype used to write the dataset should be used on the read node input.

HDF5 is able to handle partial dataset reads (e.g., a portion of an array). Version 1.0 of the toolkit supports this feature. If a non-empty array is passed to the read node input, only the elements selected will be overwritten. Read up on HDF5 dataspaces for more information.

If the read node input is not wired, the toolkit will create the best LabVIEW data type match and return it in a variant containing that type. If an empty array is passed, LVHDF5 will allocate an array which matches the dataspace of the dataset or attribute being read.

Getting Started

The best way to get started is to look at the examples in the 

<LabVIEW>\examples\UPVI\lvhdf5 directory. Start with these three:

  • basic\HDF5 Read and Write.vi
  • basic\HDF5 Read and Write with Attributes.vi
  • basic\HDF5 Logging with Unlimited Datasets.vi

then look at the rest. Also, please be sure to read up on the file format itself at www.hdfgroup.org/hdf5. It is also helpful to have HDFView, the HDF file viewer which can be downloaded from the same website.

Supported Datatypes

The following LabVIEW datatypes are supported in datasets and attributes:

  • Booleans (Stored as U8-derived committed datatype)
  • Integers (I8, I16, I32, I64, U8, U16 U32, U64)
  • Enumerations (U8, U16, U32 and U64)
  • Floating point (SGL, DBL, EXT)
  • Physical Quantity (SGL PQ, DBL PQ, EXT PQ: stored as committed datatype with unit information attached to type)
  • Complex (CSG, CDB, CXT: stored as compounds with real and imaginary members)
  • Complex Physical Quantities
  • Strings (May be stored as variable or fixed length. Defaults to variable length.)
  • Timestamps
  • HDF5 References
  • Clusters of the above types or 1-D arrays of the above types (see note below). Clusters and/or 1-D arrays may be nested recursively.

Note on array datatypes: when referring to datatypes, we are refering to the type of the element of a non-scalar dataset or attribute. The HDF5 concept of dataspace is the appropriate place to handle dimensionality of the dataset. There is no limit (beyond the HDF5 and LabVIEW imposed limits) on the rank of the dataset or dataspace. The 1-D limit is only for datatypes. Therefore, while one can store, for instance, a 4-D array of doubles in a dataset or a 4-D array of clusters containing a 1-D array, one cannot store a 1-D array of clusters where one element of the cluster is a 4-D array in a dataset. This is illustrated in the figures below.

 A 4-D array may be stored in a dataset

A 1-D array inside an array of clusters may be stored in a dataset

A 4-D array inside an array of clusters may be may not be stored. The datatype, a cluster containing a 4-D array, cannot be mapped to an HDF5 datatype.

Upgrading from the version 1.0 or 1.1 series toolkit

Minimal difficulties are expected.

Upgrading from the version 0.9 series toolkit

Little effort was made to make this toolkit backwards compatible with the version 0.9 toolkit. While the vast majority of the VIs are identical, nearly all of the really important VIs have undergone changes, or been replaced by adaptive nodes. Thus, some amount of rework will be necessary, e.g. replacing the old "Simple H5Dwrite.vi" with the new "Simple H5Dwrite" node, etc. Additional changes are also imposed in some enumerations because of the change from version 1.6 of HDF5 to version 1.8. Most data files written with the 0.9 series should be readable by the 1.0 series toolkit.

The version 1.0 series solves the inefficiency problems of the version 0.9 series of the toolkit. Version 0.9 required that all data be flattened to strings before being written to HDF5 (and read operations created flattened strings), a process which was deleterious to both memory usage and speed. Version 1.0 avoids this and works directly with the raw data.

The other major change, of course, is the use of adaptive nodes which can check the compatibility of the wired datatype at edit time rather than run time.

HDF5 Library Version

As of version 1.2.0 of Live HDF5, toolkit ships with a custom-built, thread-safe HDF5 version 1.12.0 library. You may upgrade this to later versions by replacing the files hdf5.dll and hdf5_hl.dll in the vi.lib/UPVI/lvhdf5 directory. Of course, such an upgrade is not guaranteed to function, but it probably will.