topic Is it possible to read compressed files by PC SAS on Windows 7? in SAS Programming

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Fri, 27 Jan 2012 22:10:26 GMT

Does anybody know how to read compressed files (e.g. .GZIP) using PC SAS 9.2 TS level 2M3 on Windows 7? Of course without uncompressing... It used to work on Win XP. Let's start with raw data files (text).

Is it possible to read compressed files by PC SAS on Windows 7?

art297 — Fri, 27 Jan 2012 22:20:14 GMT

You may be able to uncompress, on the fly, by piping the file in your filename statement. e.g., take a look at: http://www.ats.ucla.edu/stat/sas/faq/readgz.htm

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Fri, 27 Jan 2012 23:09:40 GMT

We used the same piping syntax in filename stmt for five years through thousands of programs and through two changes in operating systems. It stopped working after changing to Windows 7.

Is it possible to read compressed files by PC SAS on Windows 7?

FriedEgg — Tue, 31 Jan 2012 00:12:21 GMT

It seems like an installation issue with your gzip under windows 7 then...

http://gnuwin32.sourceforge.net/packages/gzip.htm

You may also try using a different utility such as 7z, which can also work with gzip files through command line and with pipes.

http://www.7-zip.org/

Re: Is it possible to read compressed files by PC SAS on Windows 7?

Doc_Duke — Tue, 31 Jan 2012 15:13:49 GMT

You haven't told us your complete environment, so these comments may not be applicable.

On Win XP (at least), piping the GZIP command produces a temporary (hidden) copy of the uncompressed data, which takes up a lot of space and disk I/O (this does not happen on *nix systems).

Windows compression gives nearly as good a compression on raw data files as GZIP.

The two of these together means that using Windows compression (and skipping the pipe) is more efficient in Windows.

------------

Caveat: I discovered the first one the hard way several years ago and can't currently remember which compression utility I was using. However, my gut feeling is that it is a Window's "feature" rather than part of gzip.

Message was edited by: Lawrence Muhlbaier

Re: Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Tue, 31 Jan 2012 16:19:43 GMT

SAS Tech support claims it is not SAS's fault. It must be the environment or the GZIP issue.

Here is the basic code.

%let File1="e:\John\ProductFiles\Product.out.20120115.gz";

%let dir=C:\"Program Files"\GnuWin32\bin\gzip;

data Product (keep=var1 var2 var3 var4);

filename file2 pipe %unquote(%str(%'&dir -cd &file1%'));

infile file2 DSD missover;

length var1 $12 var2 $14 var3 $3 var4 $20;

input var1 var2 var3 var4;

run;

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Tue, 31 Jan 2012 16:26:12 GMT

Hm, the data comes as GZIP, it will be very cumbersome to convert all the files to a different utility and ask for the input in a different format.

Is it possible to read compressed files by PC SAS on Windows 7?

FriedEgg — Tue, 31 Jan 2012 20:50:18 GMT

Try setting up your command prompt enviornment first by doing the following:

x set path=%nrstr(C:\Program Files\GnuWin32\bin;%PATH%);

*you may want to set the following to an alternate path as they are where the temporary files are written;

*by default this location is in your homedirectory, I beleive;

x set tmp=%TEMP%;

x set tmpdir=%TEMP%;

%let file1=E:\John\ProductFiles\Product.out.20120115.gz;

filename file2 pipe "gzip -cd &file1";

If you still experience issues in SAS try performing the same commands in windows command prompt and see if you experience similar issues or get an error message of some kind.

Is it possible to read compressed files by PC SAS on Windows 7?

Tom — Tue, 31 Jan 2012 21:05:19 GMT

Not a different format. Just a different utility program on you PC to read the same format.

Is it possible to read compressed files by PC SAS on Windows 7?

Tom — Tue, 31 Jan 2012 21:17:54 GMT

I doubt it is the issue, but tou should move your FILENAME statement to BEFORE the data step that is trying to use it. It is a global statement and not one that executes inside the data step.

You can simplify your FILENAME statement by using the QUOTE function rather than the macro quoting functions.

QUOTE will surround the string in dquote (") characters and double up any dquote (") characters in your generated command so that they are passed properly to the operating system.

filename file2 pipe %sysfunc(quote(&dir -cd &file1));

Try moving the quotes in your DIR (atually command)nacro variable to the front and back instead of just around Program Files part of the path.

Open a command window on your PC and run the same command to see what it does. Try piping it to more so you can scroll through the file.

> "c:\Program Files\Gnu\Win32\bin\gzip" -cd "e:\John\ProductFiles\Product.out.20120115.gz" | more

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Wed, 01 Feb 2012 03:18:05 GMT

Thank you for all your input. I really appreciate all of your answers. Since the problem occurred not on one of SAS PCs at my location, I decided to start from scratch by loading GZIP (thanks art297 and FriedEgg!) on my Win 7 machine with the same SAS installation, but no prior history of GZIP. Much to my surprise I succeeded to read a comma delimited file compressed by GZIP with no problems. So we will have to check the other PC regarding both the SAS and GZIP install.

Comments to some hints fromTom:

1) Using double quotes around the %dir path and keeping embedded space in Program Files yielded an error.

NOTE: The infile FILE2 is:

Unnamed Pipe Access Device,

PROCESS="C:\Program Files\GnuWin32\bin\gzip" -cd "c:\curr_avail.txt.gz",

RECFM=V,LRECL=256

Stderr output:

'C:\Program' is not recognized as an internal or external command,

operable program or batch file.

2) Moving the filename statement to above the data step did not make any difference, though I agree it is a global statement and should be moved up.

3) Both %unquote and %sysfunc yielded correct results.

Since I cannot replicate the general failure we'll have to test many scenarios, as FriedEgg, Doc@Duc, and Tom suggested. Unfortunately the PC in question is at a remote site for me, so it may take longer than my own fixes.

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Wed, 01 Feb 2012 03:25:39 GMT

Thanks, Doc@Duke: I may try Windows compression directly as a separate test project, but many of my input files reside on a non-Windows server. I have to copy them to my local drive.

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Wed, 01 Feb 2012 21:18:19 GMT

My apology! It turned out the PC needed a reboot after the installation of GZIP (duh!), which I did perform, but the other user did not, since the GZIP installation did not prompt for it.

Well, at least we now know that it works.

Now I have a follow up question: is it possible to read in SAS a GZIP compressed FOLDER with several text comma delimited files? If so, what would be the syntax?

Actually, when I decompress one of my standard GZ files I get two folders: one with just one file, and another one with multiple files.

Is it possible to read compressed files by PC SAS on Windows 7?

FriedEgg — Wed, 01 Feb 2012 21:21:31 GMT

GZIP files cannot contain folder structures. This must be a different type of archived file.

Is it possible to read compressed files by PC SAS on Windows 7?

Dorota_Jarosz — Wed, 01 Feb 2012 21:46:26 GMT

The filename syntax is filename.tar.gz - I assume it is GZIP. I gather the file inside is the result of applying tar archive to bundle several files, while preserving the file system. When I decompress it, I get two folders.

Is it possible to read compressed files by PC SAS on Windows 7?

FriedEgg — Wed, 01 Feb 2012 22:10:20 GMT

.tar.gz is more commonly know as a tarball. It is the result of using the tar utility and gzip. I could be incorrect but gzip alone should not be able to unpack this file, unless that is a unique feature of the program in windows vrs. unix. I would think you would need to use an alternative application such as winzip or 7-zip to handle this file as I do not this there is a dos port for tar. Once extracted you need to setup a program that will gather the file names from the extracted archive and then read those files. There are plenty of examples that show how to do this around the web, including on this forum.

Here is a link to one such paper:

Using FILEVAR= to read multiple external files in a DATA Step http://support.sas.com/techsup/technote/ts581.pdf

Re: Is it possible to read compressed files by PC SAS on Windows 7?

paulacalazans — Tue, 05 May 2015 20:46:21 GMT

HI all,

I tried to apply the code you sent but first I downloaded the Gzip:

Gzip for Windows => downloaded: Complete package, except sources

1) In the command prompt:

"C:\Users\sbrpac>"C:\Program Files (x86)\GnuWin32\bin\gzip" -cd "C:\data.csv.gz"

Returned only the first row of the csv.

2) In SAS

%let File1="C:\data.csv.gz";

%let dir=C:\"Program Files (x86)"\GnuWin32\bin\gzip;

Data result_data ;

filename file2 pipe %unquote(%str(%'&dir -cd &file1%'));

infile file2 dlm="|" LRECL=32767 truncover dsd firstobs=2;

..... run;

Returned an empty table.

Do you know what might have been wrong?

Re: Is it possible to read compressed files by PC SAS on Windows 7?

Tom — Tue, 05 May 2015 20:54:46 GMT

Two possibilities.

1) The one line is the error message from GZIP telling you why it didn't work. Try this code and check the lines in the SAS log.:

filename file2 pipe %unquote(%str(%'&dir -cd &file1%'));

Data result_data ;

infile file2 obs=2 ;

input;

put _infile_;

run;

2) The end of line characters for the CSV file do not match what your SAS session is looking for.

Try this to have SAS show you the first five lines. Any non printable characters will cause SAS to list the hex codes for the characters in the line.

filename file2 pipe %unquote(%str(%'&dir -cd &file1%'));

Data result_data ;

infile file2 obs=5 ;

input;

list;

run;

PS: You are working way too hard to manage the quotes in your OS commands. Use the QUOTE() function.

filename file2 pipe %sysfunc(quote(&dir -cd &file1));