topic Re: Read HDF5 in SAS Programming

Read HDF5

JI9 — Wed, 08 Apr 2015 21:54:21 GMT

Is there any way for SAS to read HDF5 files?

Re: Read HDF5

LinusH — Thu, 09 Apr 2015 05:52:37 GMT

By looking at the file specification, I would advise to look for an alternative way rather then trying to import it using something like INFILE.

What system/SW produced the file?

Perhaps they offer an API, like a JDBC driver.

Re: Read HDF5

jakarman — Thu, 09 Apr 2015 09:34:27 GMT

Agree with Linus, when you find a program (phyton?) there could be a way to integrated using Xcmd. XCMd is available?

Re: Read HDF5

JI9 — Thu, 09 Apr 2015 15:01:16 GMT

There is an export tool in the HDF5 application that will output csv files. And R has a package that reads HDF5. But we still use SAS for data management and I was hoping to avoid the intermediary steps. Jaap is right, though, I could use XCMD to call a Python or R routine. Thank you for your help.

Re: Read HDF5

RW9 — Thu, 09 Apr 2015 15:13:52 GMT

There is a dump tool which will export the file to XML/ASCII if that helps:

HDF5/Tools API Specification

From: http://www.hdfgroup.org/HDF5/XML/

Re: Read HDF5

JI9 — Thu, 09 Apr 2015 21:58:33 GMT

Yup, I believe that is the program R calls as part of the R package rhdf5. The solution I've settled on is calling the h5dump function in R (which in turn calls the h5dump tool you linked to, I think) from Proc IML. The bit64 package is also used because, at least in my case as I'm running 32-bit SAS 9.4, integer64 data types are created in R from the HDF5 files. SAS does not recognize the integer64 data type on SAS 9.4 32-bit, maybe 64-bit would work. Since I am getting integer counts from instruments, I was able to safely convert the integer64 data to regular integers before importing into SAS. Anyway, here is the code I've used and it appears to work.

proc iml;

submit / R;

library(rhdf5)

x <- h5dump("c:\\data\\Counts.h5",bit64conversion='bit64')

s <- do.call(cbind,x$Hist)

ss <- as.integer.integer64(s)

endsubmit;

run ImportMatrixFromR(ss,"ss");

*** Create data set work1 from the IML matrix ss. ***;

create work1 from ss;

append from ss;

close work1;

quit;