01-08-2015 11:43 AM
A SAS program I ran on PuTTY created a log file that is 4GB+. I would like to be able to read some of its contents. Its obviously too large to download from my server to my local computer. I can't open it directly on the server because, again, it is too large. How can I split this log file into many smaller files or reduce its file size so that I can read some of its contents? Thanks for any suggestions.
01-08-2015 11:56 AM
If you want to split the file you can pass the split command to your server using the appropriate OS command
ie split into 100 files:
split -l 100 my_log_file my_file
You could also consider running it through a log parser as well.
01-08-2015 12:23 PM
What happens when you try to open the log file in vi?
To transfer - zip, ftp and unzip.
1. tar -czvf filename.tar.gz filename
2. use ftp to transfer (winscp, filezilla)
3. unzip using 7-zip, winzip
I am assuming you want to transfer the file from Unix to Windows desktop.
01-08-2015 12:26 PM
Treat the log file as any external text file. Create metadata to the log file as a New ...External File...Fixed Width...and point the external file location to where your log file is . Define logical record length to 256 and select 'Pad column values with blank'. In the Column Definition, add 1 character column of length 256, begin position 1 end position 256. When you click the "Refresh" button, you should be able to see a few records (just as a test). Do not worry about setting informat. Once you have the metadata setup, you can view it (open it) as any other external file or table.
01-08-2015 04:58 PM
what I did before is to read the log file into a SAS dataset as a normal text file, Then you can do all the SAS magic to it. Here is a reference code
infile "/###/###.log" truncover dsd lrecl = 32767;
length column1 $255;
input @1 column1 $255.;
if it is 4GB, make sure you use options compress = yes.
01-09-2015 01:20 AM
Text file size can be drastically reduced with gzip and the -9 option: gzip -9 myfile
Then transfer it to your computer, and use the pipe option of the filename statement:
filename mylog pipe "gzip -dc myfile.gz";
If you're running on Windows, you can use 7zip, it has a command line interface usable within SAS via with pipe.
I do not have SAS on Windows so you may need to adapt the command line with something like
filename mylog pipe "%COMSPEC% /c 'c:\Program Files\7zip\7z mylog.gz'";
Be sure to output only what you need or you will create a huge table.
Another solution might be to filter your log file on the server with some grep commands
01-09-2015 01:21 AM
Also I have successfully used the FSLIST procedure or command with large text files. This procedure is part of Base SAS, but can only be used in an interactive SAS session on the server (via a remote login), not remotely via a PC client like EG.