topic Re: Toolkit / library for reading SAS format files (including Hebrew characters) in SAS Data Management

Toolkit / library for reading SAS format files (including Hebrew characters)

tomermahlin — Tue, 14 Nov 2017 16:57:58 GMT

What are suggested toolkits / libraries for reading SAS format files (https://v8doc.sas.com/sashtml/win/z1iles.htm) ?

I would be very much interested in getting preferable tools implemented in C++ / Java.

Preference is for open source tools.

Do you know how well they perform when data (text) in SAS format file includes Hebrew characters ?

Many thanks in advance.

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

ballardw — Tue, 14 Nov 2017 17:21:09 GMT

It might help to describe what you want to do with the format files.

Generally everything you might want to know would likely best come from Proc Format unless you do not use SAS at all.

Proc Format with the page option will create a document of values involved or use CNTLOUT to create a SAS dataset.

Appearance of results could depend on the LOCALE setting of the SAS system for language support.

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

tomermahlin — Tue, 14 Nov 2017 18:17:38 GMT

I attach a file as an example of SAS format file which is not properly read by 3rd party library.

Hebrew data / text seems to be encoded with ISO 8859-8 code page in this file.

SPSS Modeler leveraging this 3rd party library is not capable of properly processing Hebrew data (as a result of faulty SAS reading engine in this 3rd party library).

Obviously one of the solutions would be to seek resolution of proper support for Hebrew in 3rd party library. However, usually embarking on this journey may be very time consuming and not necessarily productive.

Thus I was wondering if there is any preferable / recommend toolkit / library for reading SAS files. By a way of example:

for handling / reading PDF documents I would suggest to use: https://pdfbox.apache.org/
for handling / reading Microsoft Office documents I would suggest to use: https://poi.apache.org/

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

Reeza — Tue, 14 Nov 2017 22:13:03 GMT

V8 files are almost 20 years old. Getting them to work may be problematic...is this reflective of your actual situation?

Third party library are maintained by the developer so you should reach out to the developer. SAS file formats are proprietary and as far as I know they do not provide tools to read them for other applications.

R has the SAS7BDAT packages that reads SAS files, but not sure it has anything to read a format file, which is actually a catalog (I think).

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

tomermahlin — Wed, 15 Nov 2017 14:03:18 GMT

Yes. This is reflective of actual situation. Customer are trying to work with files (similar to what I attached) using SPSS Modeler.

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

Reeza — Wed, 15 Nov 2017 16:08:01 GMT

They are SAS files, not format files or such which is a different story.

You can try the SAS Universal Viewer - download from SAS to read the file and then perhaps export as CSV or such.

Or you can try STAT/Transfer which is a relatively cheap third party tool to read SAS files.

Re: Toolkit / library for reading SAS format files (including Hebrew characters)

tomermahlin — Wed, 15 Nov 2017 16:58:09 GMT

STAT/Transfer tool (library DLL) from https://stattransfer.com/downloads/ is exactly what is being used by SPSS Modeler :-)).

This tool has issues with Hebrew code page.

I tried SAS Universal Viewer (from https://support.sas.com/downloads/package.htm?pid=1989#) and got very good results (see attached image illustrating how file looks like in SAS Universal Viewer vs. SPSS Modeler).

PS. When I mentioned a word "format" I referred to internal structure of SAS binary files.

It is described here: https://cran.r-project.org/web/packages/sas7bdat/vignettes/sas7bdat.pdf

I realize that there might be several different formats used by SAS Inc.

For the sake of this specific discussion sas7bdat format seems to be most relevant.