topic Re: import txt file compressed into .ZIP or .Gz or .7z in SAS Programming

import txt file compressed into .ZIP or .Gz or .7z

sas_lak — Fri, 19 Dec 2014 11:10:18 GMT

Hello All,

How can i import the txt file which is compressed into .ZIP or .Gz or .7z

using infile statement from (windows) SAS 9.2 version.

Re: import txt file compressed into .ZIP or .Gz or .7z

RW9 — Fri, 19 Dec 2014 11:20:49 GMT

You need to decompress the file first, there is no functionality at this time to unzip within SAS. You can either do it manually when you receive the file, or pipe an OS command out using x or call system, e.g.:

data _null_;

winzip_path="'c:\programs\winzip\winzip.exe'"; /* Note the double quotes around single quotes */

filepath="'s:\zips\myzip.zip'";

cmd=winzip_path||" "||filepath;

x cmd;

run;

Re: import txt file compressed into .ZIP or .Gz or .7z

sas_lak — Fri, 19 Dec 2014 11:30:14 GMT

Thanks for quik reply, but every day we received number of files and each compressed file having minimum 5GB,

is there any other option to import into SAS.

Re: import txt file compressed into .ZIP or .Gz or .7z

RW9 — Fri, 19 Dec 2014 11:38:53 GMT

Why not run a script in your OS to unzip the files. They will need to be unzipped to be used.

Re: import txt file compressed into .ZIP or .Gz or .7z

Kurt_Bremser — Fri, 19 Dec 2014 14:01:41 GMT

filename oscmd pipe "gzip -dc &infilename";

data import;

infile oscmd;

/* process infile as if unzipped */

run;

This will stream the unzipped contents of the file specified in infilename to your data step.

Consult the proper manuals for other compressed file formats and the correpsonding utilities.

Re: import txt file compressed into .ZIP or .Gz or .7z

Patrick — Sat, 20 Dec 2014 00:51:16 GMT

You can unzip & read the files "on-the-fly" with syntax as suggests.

With SAS 9.4 there is now also a zip engine available which would allow you to read such archives

SAS(R) 9.4 Statements: Reference, Third Edition

Re: import txt file compressed into .ZIP or .Gz or .7z

Babloo — Sat, 20 Dec 2014 04:53:31 GMT

May I request you to explain the "gzip -dc &infilename" from your filename statement?

Thanks.

Re: import txt file compressed into .ZIP or .Gz or .7z

Patrick — Sat, 20 Dec 2014 07:52:31 GMT

I suggest that you do a Google search using "site:support.sas.com filename pipe". Also searching here in this forum with terms like "filename pipe zip" will return quite a few links.

"gzip -dc &infilename"

gzip -dc Is calling the 3rd party GZIP tool with appropriate switches for this tool

&infilename A SAS macro variable which you could define and populate earlier in your code. This is the path and name of your GZIP archive which you pass in as parameter into GZIP

The example is for GZIP. What's the right 3rd party tool for your archive will depend on what's available in your specific environment.

Re: import txt file compressed into .ZIP or .Gz or .7z

Kurt_Bremser — Mon, 22 Dec 2014 06:39:03 GMT

gzip:

-d: decompresses the file (without the -d gzip will do a compression)

-c: instead of writing a compressed/decompressed file, stream the output to stdout

The filename pipe works similar to the reverse apostroph (backtick) under UNIX. The string is executed, and the output is provided to SAS as if you read a file.

The filename pipe is also very useful if you want to include the output and error messages of a command in the SAS log:

filename oscmd pipe "what_you_want_to_execute 2>&1"; * 2>&1 redirects stderr to stdout, so you get everything;

data _null_;

infile oscmd;

input;

put _infile_; * write the whole input line to the log;

run;

As Patrick suggested, you may need to adapt the command to the zip tool available in your environment. All those tools should have options for batch processing.

Re: import txt file compressed into .ZIP or .Gz or .7z

sas_lak — Mon, 22 Dec 2014 07:22:17 GMT

Thank you one and all,

Here I am receiving minimum 50 compressed files in a week time, and it contains approximate 100 GB.

Every time I used to uncompressed and import flat files into SAS. It consumes huge disc space.

Any way am deleting the all files, once SAS datasets are placed into my libelers.

Thank you very much for your suggestions.

Re: import txt file compressed into .ZIP or .Gz or .7z

Patrick — Mon, 22 Dec 2014 10:18:46 GMT

Content of post deleted as wrong and being convinced otherwise by

Re: import txt file compressed into .ZIP or .Gz or .7z

Kurt_Bremser — Mon, 22 Dec 2014 11:59:00 GMT

Actually, gzip -dc will not use any diskspace at all. So the pipe approach as described only needs the space for the zipped file and the space for the SAS dataset.