topic Re: GPG Files Readin in SAS in SAS Programming

Read in GPG Encrypted Files in SAS

Konakanchi — Tue, 15 May 2018 15:22:48 GMT

I have a file test.txt.gpg, a text file encrypted with a pass phrase. How should I readin in SAS? I have gone through other posts and found that there is a way to decrypt first using X command and then readin. Need more inputs

Re: GPG Files Readin in SAS

SimonDawson — Tue, 15 May 2018 12:06:01 GMT

X command simply runs the syntax you provide it on the system shell.
When it comes to what you need to put into the X statement please refer to the gnupg documentation.
https://www.gnupg.org/documentation/manuals.html
If you can decrypt it in a command prompt, then what you run in there is what you are going to put inside your X statement inside SAS.

Once you have the document decrypted you'd read it like any other raw data file into SAS.

Re: GPG Files Readin in SAS

Konakanchi — Tue, 15 May 2018 12:15:28 GMT

My syntax looks like

%MACRO DATA_READIN(LIB,FILE,DSN);

* Supplying the data to SAS through PIPE, as the files are compressed ;

FILENAME READER PIPE "M:\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;
INPUT

I am able to decrypt the file through this but the program is end. The next lines of program aren't running.

Log says

NOTE: The infile READER is:
Unnamed Pipe Access Device,
PROCESS=M:\Rawdata\Test1.txt.gpg,
RECFM=V,LRECL=32767

NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T1 has 0 observations and 69 variables.
NOTE: DATA statement used (Total process time):
real time 6.36 seconds
cpu time 0.04 seconds

Re: GPG Files Readin in SAS

Kurt_Bremser — Tue, 15 May 2018 12:23:07 GMT

@Konakanchi wrote:

My syntax looks like

%MACRO DATA_READIN(LIB,FILE,DSN);

* Supplying the data to SAS through PIPE, as the files are compressed ;

FILENAME READER PIPE "M:\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;
INPUT

I am able to decrypt the file through this but the program is end. The next lines of program aren't running.

Log says

NOTE: The infile READER is:
Unnamed Pipe Access Device,
PROCESS=M:\Rawdata\Test1.txt.gpg,
RECFM=V,LRECL=32767

NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T1 has 0 observations and 69 variables.
NOTE: DATA statement used (Total process time):
real time 6.36 seconds
cpu time 0.04 seconds

That happens because M:\Rawdata\&FILE..txt.gpg is just a file and not something that can be run on the system.

In filename pipe, you need to supply a commandline that decrypts the file and writes the decrypted contents to standard output (which is what you then read in SAS).

So you need to first find out (see the gnupg documentation referenced earlier by @SimonDawson) how to do that.

Re: GPG Files Readin in SAS

Konakanchi — Tue, 15 May 2018 12:41:06 GMT

Yep.

I have changed it to

FILENAME READER PIPE "gpg --decrypt-files &PATH.\Rawdata\&FILE..txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1;

I see that the file is decrypted in my folder but no SAS is not reading in.

Log says

Stderr output:
gpg: AES encrypted data
gpg: encrypted with 1 passphrase
NOTE: 0 records were read from the infile READER.
NOTE: The data set RSASDATA.T3 has 0 observations and 69 variables.

Re: GPG Files Readin in SAS

SimonDawson — Wed, 16 May 2018 04:28:41 GMT

The --decrypt-files option doens't write clear text into the stdout. Its working with files.

# Make a clear text message in a file
$ echo message1 > clear1

# Encrypt it
$ gpg -c clear1

# Decrypt it
$ gpg --decrypt-files clear1.gpg
gpg: AES encrypted data
gpg: encrypted with 1 passphrase
File 'clear1' exists. Overwrite? (y/N)
Enter new filename:
gpg: signal Interrupt caught ... exiting

Note the gpg command is prompting and working with files. Its not piping the cleartext message into stdout which is what SAS is expecting to read from when you use a pipe.

If you want to do it with files then run the GPG decrypt with an X statement and read the cleartext from the output file in the normal way.

This isn't any issue with your SAS code but rather what you have typed into the filename pipe statement to be run. You'll need to work out what options gnupg needs on the command line to stream the clear text into stdout from cipher text stored on disk.

Re: GPG Files Readin in SAS

Kurt_Bremser — Wed, 16 May 2018 06:10:59 GMT

if you use

--decrypt file.gpg

instead of

--decrypt-files file.gpg

the result will be written to stdout, so you can capture it with the filename pipe.

(retrieved from the first hit of a google search for "man gpg"; Maxim 1 is also valid for UNIX commands)

Re: GPG Files Readin in SAS

Konakanchi — Thu, 17 May 2018 06:39:21 GMT

Worked. Thanks. However, I have another doubt. My gpg file as a gzip file , within which there is a text file. The text file needs to be imported into SAS. My code which has worked based on your suggestions to read a pgp encrypted text file was

FILENAME READER PIPE "gpg --decrypt &PATH.\Rawdata\Test3.txt.gpg";

DATA &LIB..&DSN. (COMPRESS=YES) ;
%LET _EFIERR_ = 0; /* set the ERROR detection macro variable */
INFILE READER DELIMITER = '|' TRUNCOVER DSD LRECL=32767 FIRSTOBS=1 OBS=10;

Where can i add the command to extract text file from gzip here?

Re: GPG Files Readin in SAS

Kurt_Bremser — Thu, 17 May 2018 07:02:01 GMT

So you have a file that was probably created with something like

gzip -c file.txt|gpg>file.txt.gpg

(I've omitted the gpg options for accepting stdin and writing to stdout)

So it is text compressed with gzip encrypted with gpg.

To read that, you need to unravel everything in the reverse order of its creation:

filename reader pipe "gpg --decrypt &PATH.\Rawdata\Test3.txt.gpg|gzip -dc";

So the decrypted stream is fed to gzip, which is told by the parameters to decompress and write to standard output; since no filename is given, gzip will read from stdin and therefore the pipe from gpg.

An alternative would be to decrypt to a file first, and then read that with filename zip with the gzip option.