Desktop productivity for business analysts and programmers

Reading from Unix

Accepted Solution Solved
Reply
Contributor
Posts: 46
Accepted Solution

Reading from Unix

Using EG 5.1.

 

I'm trying to read from a compressed Unix file using the following code

 

FILENAME Name pipe "zcat sas-data/SAD/SAD/IFRS 9/BASES/File_name.Z";

Data Test;

infile Name

lrecl=500000

DLM=':'

missover DSD;

label;

Run;

 

But the output comes out looking like this:

empty.jpg

 

Unziping the file and opening it directly, it looks like this:

42000001251100:137224761:123272858:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000Smiley Very HappyW7 Smiley Tongue:CP :+002296660:+00000005300:+00000598760:+00000132500:+00000132500:+00000000000:+00000250000:   :00010101:ENC:20120314:ENC:20120215:0:502:00010101:20100914:20111027:+00000007300:119
42000002992100:176705643:184314704:+00000188769:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000Smiley TongueW7 Smiley Very Happy:CP :+002123330:+00000008000:+00001768510:+00000200000:+00000200000:+00000001389:+00001566513:60:99991231:ENC:20150323:ENC:20150223:0:502:00010101:20141008:20130531:+00000005197:113
42000003721100:134284160:134284151:+00000138210:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000:+00000000000Smiley TongueW7 Smiley Very Happy:CP :+002296660:+00000009000:+00001297485:+00000225000:+00000225000:+00000000970:+00000739335:68:20140114:ENC:20140704:RET:20140702:0:502:00010101:20121211:00010101:+00000000000:113

 

How can I make this work?


Accepted Solutions
Solution
‎10-28-2016 06:45 AM
Contributor
Posts: 46

Re: Reading from Unix

FILENAME Name pipe "zcat sas-data/SAD/SAD/IFRS 9/BASES/File_name.Z";

Data Test;

infile Name

lrecl=500000

DLM=':'

missover DSD;

Input Var1-Var36;

Run;

The input statement was missing. The above fixes it.

View solution in original post


All Replies
Regular Contributor
Posts: 154

Re: Reading from Unix

I see you have a space in the directory "IFRS 9" in the file path. Can you escape it with a backslash ("IFRS\ 9") and try again?

Regular Contributor
Posts: 154

Re: Reading from Unix

Also, that's a relative path... can you give it the full path, e.g. /home/user/username/...
Esteemed Advisor
Posts: 6,706

Re: Reading from Unix

I see two issues here

- a relative path that probably won't work, because SAS will go looking for the file from the current directory, which is somewhere in the SAS configuration tree if the default setting has not been changed.

- a blank in the filename

BLANKS IN FILE OR PATH NAMES (on servers) ARE A BAD IDEA (repeat this 1k times)

 

Try this:

FILENAME Name pipe 'zcat "/sas-data/SAD/SAD/IFRS 9/BASES/File_name.Z"';

If the compressed file is actually somewhere up from your home directory, use $HOME instead of the leading slash.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 46

Re: Reading from Unix

I physically changed the path to remove the space and I tried FILENAME Name pipe 'zcat "/sas-data/SAD/SAD/IFRS9/BASES/File_name.Z"', but it still doesn't work. This particular line of code does run, it also ran in my original try, but when reading from the file it comes out empty. Any ideas?
Esteemed Advisor
Posts: 6,706

Re: Reading from Unix

Then try this:

FILENAME Name pipe 'zcat "/sas-data/SAD/SAD/IFRS9/BASES/File_name.Z" 2>&1';

data _null_;
infile name;
input;
put _infile_;
if _n_ > 10 then stop;
run;

That should at least put error messages from the external command into the log.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 46

Re: Reading from Unix

17 data _null_;
18 infile name;
19 input;
20 put _infile_;
21 if _n_ > 10 then stop;
22 run;

NOTE: The infile NAME is:
Pipe command="zcat "/sas-data/SAD/SAD/IFRS9/BASES/File_name.Z" 2>&1"

gzip: /sas-data/SAD/SAD/IFRS9/BASES/File_name.Z: No such file or directory
NOTE: 1 record was read from the infile NAME.
The minimum record length was 74.
The maximum record length was 74.

I managed to make it work, though. I was missing the input option. Should I post what worked for me as an answer or what?
Esteemed Advisor
Posts: 6,706

Re: Reading from Unix


sleretrano wrote:
17 data _null_;
18 infile name;
19 input;
20 put _infile_;
21 if _n_ > 10 then stop;
22 run;

NOTE: The infile NAME is:
Pipe command="zcat "/sas-data/SAD/SAD/IFRS9/BASES/File_name.Z" 2>&1"

gzip: /sas-data/SAD/SAD/IFRS9/BASES/File_name.Z: No such file or directory
NOTE: 1 record was read from the infile NAME.
The minimum record length was 74.
The maximum record length was 74.

I managed to make it work, though. I was missing the input option. Should I post what worked for me as an answer or what?

Oh, gosh. Should have noticed that missing input statement.

 

By all means, post the final solution for future reference.

 

The 2>&1 redirection of stderr to stdout is one of the most useful tricks when using external commands. I basically never use the x statement any longer in production jobs, I always use filename pipe with redirection, so I get error messages in the log or in files.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 46

Re: Reading from Unix

No idea of what you're talking about. Can you please refer me to some intro to that?
Thank you for everything.
Solution
‎10-28-2016 06:45 AM
Contributor
Posts: 46

Re: Reading from Unix

FILENAME Name pipe "zcat sas-data/SAD/SAD/IFRS 9/BASES/File_name.Z";

Data Test;

infile Name

lrecl=500000

DLM=':'

missover DSD;

Input Var1-Var36;

Run;

The input statement was missing. The above fixes it.

Esteemed Advisor
Posts: 6,706

Re: Reading from Unix

So the compressed data file really resides in a path relative to the current working directory of the SAS workspace server process.

And it worked just like that, with an unmasked blank in the file name?

I ask that because on my UNIX that results in zcat trying to unzip two separate files. From the commandline and from SAS.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 46

Re: Reading from Unix

Yes, it worked.
I've always heard spaces are a bad idea, but I've never had an issue with them, to be honest.
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 11 replies
  • 562 views
  • 3 likes
  • 3 in conversation