DATA Step, Macro, Functions and more

Matter with unzipping a big file

Reply
Super Contributor
Posts: 371

Matter with unzipping a big file

[ Edited ]

Hello,

 

I have a big file .zip :  Y:/M/N/BIG_File.zip, the size of big_file.zip is 6 GB.

So I want to unzip the file big_file.zip  into a new folder : BIG_FILE

 rep --> Y:/M/N/rep

 

I use this sas code

 

%let dir=Y:/M/N;

X cd &dir;

filename pipe "unzip  BIG_FILE -d  REP/BIG_FILE";

 

So I get this message :

unzip : cannot find  BIG_FILE.zip, BIG_FILE.ZIP, BIG_FILE

 

So If i use the last code for a small file, the unzip works well.

So I need to unzip a big directory with more than 6 GB.

Thank you

 

 

 

 

 

 

 

Super User
Posts: 5,071

Re: Matter with unzipping a big file

Could it just be a spelling issue?  Unix treats upper case as different than lower case.

Super Contributor
Posts: 371

Re: Matter with unzipping a big file

[ Edited ]

Thank you for your intereste.

No, it is not a matter of spelling,lowwer case, or upper case.

I have tested that.

the unzip works well for the small file, but not for a big file

 

 

Super User
Posts: 17,750

Re: Matter with unzipping a big file

Post your log a screenshot of your file in the path please.

 

Also, does it work if you do it via command line? If not, then it's not a SAS issue, it's an issue with the zip utility. 

Super Contributor
Posts: 371

Re: Matter with unzipping a big file

Thank you for your answer.

I will check that

Super User
Super User
Posts: 6,497

Re: Matter with unzipping a big file

What version of ZIP does your system have?  You might not have support for large files.  You need at least ZIP 3.0.  Or you could try 7ZIP instead.

Super Contributor
Posts: 371

Re: Matter with unzipping a big file

@Tom: Thank your message.

I think gzip. But as I hear from some people, sas used his zip.

So how can I identify the zip utilities on unix  and version?

Super User
Posts: 17,750

Re: Matter with unzipping a big file

@LineMoon The method you're using to unzip is not a SAS method or procedure. It's using SAS to pass Unix Commands to your unix system. So whatever is on your box is what you have available. 

 

 

It's another way of saying:

 

X "unzip blabhlbhafile";

 

If you want to know the utilities on unix ask your administrator. 

Super User
Posts: 6,928

Re: Matter with unzipping a big file

SAS has a built-in engine to unzip input files on the fly, which you can use in filename and infile statements. Things that you call externally, OTOH, depend on the commands supplied by the operating system.

 

By the way, you used

%let dir=Y:/M/N;

which looks like a Windows path, as UNIX does not use (and does not allow) drive letters.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Super Contributor
Posts: 371

Re: Matter with unzipping a big file

[ Edited ]

No , I want to say this

%let dir=Y/M/N;

No , it is not a windows path, it is a unix path.

in my code, I have used the correct path in unix   %let dir= Y/M/N;

 As I say before, the code does the job well for a small file, but it does not work for a big file ( more than 2 GB)

 

%let dir= Y/M/N;

X cd &dir;

filename pipe "unzip  BIG_FILE -d  REP/BIG_FILE";

Super User
Posts: 6,928

Re: Matter with unzipping a big file

Don't do a separate cd; instead use fully qualified filenames in the filename pipe. An external change directory will most probably have no effect on the current working directory of the SAS session.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Super Contributor
Posts: 371

Re: Matter with unzipping a big file

[ Edited ]

Thank you for your answer.

Please,

what is the impact of using "separate CD" in that ?

Can you explain " use fully qualified filenames in the filename pipe" ?

 

I observe that, If I do not use a cd, I get a strainge result for a small file, but it does not work for a big file

 

filename pipe "unzip V1/V2/V3/W1/W2  SMALL_FILE -d  V1/V2/R/L/Small_FILE";

 

Is it possible to unzip a big file by sas without using Unix command.

 

Thank you again

 

Super User
Posts: 17,750

Re: Matter with unzipping a big file

Does the line work outside of SAS, can you unzip the file successfully at all?

Super Contributor
Posts: 371

Re: Matter with unzipping a big file

Thank you for your answer.

I will check.

For information, the line command works for a small file in sas

Super User
Posts: 6,928

Re: Matter with unzipping a big file

First of all, just read the documentation: http://support.sas.com/documentation/cdl/en/lestmtsref/68024/HTML/default/viewer.htm#n1dn0f61yfyzton...

The ZIP Access Method only works for files compressed with WinZip.

What's so bad about using the right tool for unzipping files? UNIX features all tools necessary for uncompressing, so use them.

 

Regarding the "separate cd": UNIX keeps the current working directory in an environment variable (PWD) of the current process. When you use cd in a separate step, you might just change the PWD there, but (in a later step) end up in the initial working directory of the SAS process, as the environment of the subprocess where the cd was performed is lost. Therefore I recommend to use fully qualified path names for the files in the unzip command. A fully qualified path name is one that starts at the root directory - with a slash.

Never assume, take full control.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Ask a Question
Discussion stats
  • 17 replies
  • 401 views
  • 6 likes
  • 5 in conversation