BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Konakanchi
Calcite | Level 5

I have a file test.txt.gpg, a text file encrypted with a pass phrase. How should I readin in SAS? I have gone through other posts and found that there is a way to decrypt first using X command and then readin. Need more inputs

1 ACCEPTED SOLUTION

Accepted Solutions
Kurt_Bremser
Super User

if you use

--decrypt file.gpg

instead of

--decrypt-files file.gpg

the result will be written to stdout, so you can capture it with the filename pipe.

(retrieved from the first hit of a google search for "man gpg"; Maxim 1 is also valid for UNIX commands)

View solution in original post

8 REPLIES 8
SimonDawson
SAS Employee
X command simply runs the syntax you provide it on the system shell.
When it comes to what you need to put into the X statement please refer to the gnupg documentation.
https://www.gnupg.org/documentation/manuals.html
If you can decrypt it in a command prompt, then what you run in there is what you are going to put inside your X statement inside SAS.

Once you have the document decrypted you'd read it like any other raw data file into SAS.
Konakanchi
Calcite | Level 5

My syntax looks like 

 

%MACRO DATA_READIN(LIB,FILE,DSN);

* Supplying the data to SAS through PIPE, as the files are compressed ;

FILENAME READER PIPE "M:\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;
INPUT

 

I am able to decrypt the file through this but the program is end. The next lines of program aren't running.

 

Log says 

 

NOTE: The infile READER is:
Unnamed Pipe Access Device,
PROCESS=M:\Rawdata\Test1.txt.gpg,
RECFM=V,LRECL=32767

NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T1 has 0 observations and 69 variables.
NOTE: DATA statement used (Total process time):
real time 6.36 seconds
cpu time 0.04 seconds

Kurt_Bremser
Super User

@Konakanchi wrote:

My syntax looks like 

 

%MACRO DATA_READIN(LIB,FILE,DSN);

* Supplying the data to SAS through PIPE, as the files are compressed ;

FILENAME READER PIPE "M:\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;
INPUT

 

I am able to decrypt the file through this but the program is end. The next lines of program aren't running.

 

Log says 

 

NOTE: The infile READER is:
Unnamed Pipe Access Device,
PROCESS=M:\Rawdata\Test1.txt.gpg,
RECFM=V,LRECL=32767

NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T1 has 0 observations and 69 variables.
NOTE: DATA statement used (Total process time):
real time 6.36 seconds
cpu time 0.04 seconds


That happens because M:\Rawdata\&FILE..txt.gpg is just a file and not something that can be run on the system.

In filename pipe, you need to supply a commandline that decrypts the file and writes the decrypted contents to standard output (which is what you then read in SAS).

So you need to first find out (see the gnupg documentation referenced earlier by @SimonDawson) how to do that.

Konakanchi
Calcite | Level 5

Yep.

 

I have changed it to 

 

FILENAME READER PIPE "gpg --decrypt-files &PATH.\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;

I see that the file is decrypted in my folder but no SAS is not reading in. 

 

Log says

 

Stderr output:
gpg: AES encrypted data
gpg: encrypted with 1 passphrase
NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T3 has 0 observations and 69 variables.

SimonDawson
SAS Employee
The --decrypt-files option doens't write clear text into the stdout. Its working with files.

# Make a clear text message in a file
$ echo message1 > clear1

# Encrypt it
$ gpg -c clear1

# Decrypt it
$ gpg --decrypt-files clear1.gpg
gpg: AES encrypted data
gpg: encrypted with 1 passphrase
File 'clear1' exists. Overwrite? (y/N)
Enter new filename:
gpg: signal Interrupt caught ... exiting


Note the gpg command is prompting and working with files. Its not piping the cleartext message into stdout which is what SAS is expecting to read from when you use a pipe.

If you want to do it with files then run the GPG decrypt with an X statement and read the cleartext from the output file in the normal way.

This isn't any issue with your SAS code but rather what you have typed into the filename pipe statement to be run. You'll need to work out what options gnupg needs on the command line to stream the clear text into stdout from cipher text stored on disk.
Kurt_Bremser
Super User

if you use

--decrypt file.gpg

instead of

--decrypt-files file.gpg

the result will be written to stdout, so you can capture it with the filename pipe.

(retrieved from the first hit of a google search for "man gpg"; Maxim 1 is also valid for UNIX commands)

Konakanchi
Calcite | Level 5

Worked. Thanks. However, I have another doubt. My gpg file as a gzip file , within which there is a text file. The text file needs to be imported into SAS. My code which has worked based on your suggestions to read a pgp encrypted text file was

 

FILENAME READER PIPE "gpg --decrypt &PATH.\Rawdata\Test3.txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1 OBS=10;

 

Where can i add the command to extract text file from gzip here?

Kurt_Bremser
Super User

So you have a file that was probably created with something like

gzip -c file.txt|gpg>file.txt.gpg

(I've omitted the gpg options for accepting stdin and writing to stdout)

So it is text compressed with gzip encrypted with gpg.

To read that, you need to unravel everything in the reverse order of its creation:

filename reader pipe "gpg --decrypt &PATH.\Rawdata\Test3.txt.gpg|gzip -dc";

So the decrypted stream is fed to gzip, which is told by the parameters to decompress and write to standard output; since no filename is given, gzip will read from stdin and therefore the pipe from gpg.

 

An alternative would be to decrypt to a file first, and then read that with filename zip with the gzip option.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 8 replies
  • 2379 views
  • 0 likes
  • 3 in conversation