DATA Step, Macro, Functions and more

import txt file compressed into .ZIP or .Gz or .7z

Reply
Contributor
Posts: 46

import txt file compressed into .ZIP or .Gz or .7z

Hello All,

How can i import the txt file which is compressed into    .ZIP or .Gz or .7z

using infile statement from (windows) SAS 9.2 version.

Super User
Super User
Posts: 7,942

Re: import txt file compressed into .ZIP or .Gz or .7z

You need to decompress the file first, there is no functionality at this time to unzip within SAS.  You can either do it manually when you receive the file, or pipe an OS command out using x or call system, e.g.:

data _null_;

     winzip_path="'c:\programs\winzip\winzip.exe'";  /* Note the double quotes around single quotes */

     filepath="'s:\zips\myzip.zip'";

     cmd=winzip_path||" "||filepath;

     x cmd;

run;

Contributor
Posts: 46

Re: import txt file compressed into .ZIP or .Gz or .7z

Thanks for quik reply, but every day we received number of files and each compressed file having minimum 5GB,

is there any other option to import into SAS.

Super User
Super User
Posts: 7,942

Re: import txt file compressed into .ZIP or .Gz or .7z

Why not run a script in your OS to unzip the files.  They will need to be unzipped to be used.

Respected Advisor
Posts: 4,173

Re: import txt file compressed into .ZIP or .Gz or .7z

You can unzip & read the files "on-the-fly" with syntax as suggests.

With SAS 9.4 there is now also a zip engine available which would allow you to read such archives

SAS(R) 9.4 Statements: Reference, Third Edition

Super User
Posts: 7,760

Re: import txt file compressed into .ZIP or .Gz or .7z

filename oscmd pipe "gzip -dc &infilename";

data import;

infile oscmd;

/* process infile as if unzipped */

run;

This will stream the unzipped contents of the file specified in infilename to your data step.

Consult the proper manuals for other compressed file formats and the correpsonding utilities.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Super Contributor
Posts: 435

Re: import txt file compressed into .ZIP or .Gz or .7z

Posted in reply to KurtBremser

May I request you to explain the "gzip -dc &infilename" from your filename statement?

Thanks.

Respected Advisor
Posts: 4,173

Re: import txt file compressed into .ZIP or .Gz or .7z

I suggest that you do a Google search using "site:support.sas.com filename pipe". Also searching here in this forum with terms like "filename pipe zip" will return quite a few links.

"gzip -dc &infilename"


gzip -dc           Is calling the 3rd party GZIP tool with appropriate switches for this tool

&infilename     A SAS macro variable which you could define and populate earlier in your code. This is the path and name of your GZIP archive which you pass in as parameter into GZIP


The example is for GZIP. What's the right 3rd party tool for your archive will depend on what's available in your specific environment.

Super User
Posts: 7,760

Re: import txt file compressed into .ZIP or .Gz or .7z

gzip:

-d: decompresses the file (without the -d gzip will do a compression)

-c: instead of writing a compressed/decompressed file, stream the output to stdout

The filename pipe works similar to the reverse apostroph (backtick) under UNIX. The string is executed, and the output is provided to SAS as if you read a file.

The filename pipe is also very useful if you want to include the output and error messages of a command in the SAS log:

filename oscmd pipe "what_you_want_to_execute 2>&1"; * 2>&1 redirects stderr to stdout, so you get everything;

data _null_;

infile oscmd;

input;

put _infile_; * write the whole input line to the log;

run;

As Patrick suggested, you may need to adapt the command to the zip tool available in your environment. All those tools should have options for batch processing.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 46

Re: import txt file compressed into .ZIP or .Gz or .7z

Posted in reply to KurtBremser

Thank you one and all,

Here I am receiving minimum 50 compressed files in a week time, and it contains approximate 100 GB.

Every time I used to uncompressed and import flat files into SAS. It consumes huge disc space.

Any way am deleting the all files, once SAS datasets are placed into my libelers.

Thank you very much for your suggestions.

Respected Advisor
Posts: 4,173

Re: import txt file compressed into .ZIP or .Gz or .7z

Content of post deleted as wrong and being convinced otherwise by

Super User
Posts: 7,760

Re: import txt file compressed into .ZIP or .Gz or .7z

Actually, gzip -dc will not use any diskspace at all. So the pipe approach as described only needs the space for the zipped file and the space for the SAS dataset.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Ask a Question
Discussion stats
  • 11 replies
  • 3825 views
  • 15 likes
  • 5 in conversation