BookmarkSubscribeRSS Feed
jhaque
Calcite | Level 5

Hi All,

Can you please help on below points. Your comments highly appreciated.

1. Using proc export xx.dat file is created and put into unix box. Post creating xx.dat file How to take record count and add this value to the first line in the xx.dat file using unix commands in sas program.

2. Using datastep reading data from yyy.csv file. The yyy.csv file having record count (already provided say RC=100000) in the first line. How to fetch that record count(RC) and check the record count is matched with the present total number of records the only create dataset from yyy.csv file otherwise raise an error.It should be done using unix commands in sas program.

10 REPLIES 10
Doc_Duke
Rhodochrosite | Level 12

JHaque,

Not enough information, but you are probably asking this in the wrong place.

What you are asking about is how to write a unix shell script and launch it from SAS.  The launching from SAS is easy, use the "x" command (Easy, that is, unless you SAS Admin has disabled it.).

How you write your shell scripts is beyond this forum.  Try a unix forum for your flavor of *nix (there are subtle, but often infuriating, differences among them).  You will need to provide them with the name of the shell you are using (probably "bash").  You may also need Perl available for some of the string processing.

(I know just enough shell scripting to be dangerous, so I won't attempt a direct answer to your questions.)

Doc Muhlbaier

Duke

tlk
Quartz | Level 8 tlk
Quartz | Level 8

JHaque,

You could use the Unix command wc in something like this:

filename entry pipe "wc -l xx.dat";

data _null_;

    infile entry;

        input lcount 8.; /** read the result of the wc command **/

    call symput('lcountM',lcount); /** create a macro variable **/

run;

After you could compare the result of wc command (&lcountM) with the number of observation in your sas dataset.

Laurent

SASKiwi
PROC Star

In my experience, it is more common to write a trailer record to an exported text file with the row rount recorded, with the header record containing column names. If you take this approach then you don't need to use Unix commands, you can write the custom export in a SAS DATA step and SAS can write the trailer record with the row count.

AndrewHowell
Moderator

In both cases, I don't see why Unix commands are needed, everything should be able to be done from within SAS.

1. PROC EXPORT won't help here; the simplicity of most PROC STEPS means some specific requests (like this) can't be done that way. Alternative, using DATA STEP, which also does not require Unix commands, plus it means you are not unneccesarily reprocessing either the source data set or the target DAT file:

filename MyOut "./xx.dat";

data _null_;

  file MyOut; /* Insert options of choice here */

  set MyInputTable nobs=RecordCount end=IsLast;

  if _N_ eq 1 then do;

    /* All your header stuff goes here */

    put "Records: " RecordCount;

  end;

  /* Output each line, apply formats as desired */

  put _ALL_;

  if IsLast then do;

    /* All your footer stuff goes here (if any) */

  end;

run;

filename MyOut clear;

2. Similar DATA STEP methodology applies here:

filename MyInFile "./yy.csv";

data MyOutput;

  infile MyInFile dsd end=IsLast; /* Insert other options of choice here */

  if _N_ eq 1 then do;

    input RecordCount; /* Syntax of input statement depends on the layout of text file */

    put "Records: " RecordCount;

  end;

/*

  Accrue an incremental count for every records read - ActualCount.

  NOTE: I don't know if your first line also contains data, therefore

  do not know if the next line should or shouldn't be within an "ELSE"

  associated with the above "IF". Plus this may affect whether you are

  comparing ActualCount with RecordCount, or ActualCount with (RecordCount-1)

*/

  input Var1 Var2 Var3; /* Syntax of input statement depends on the layout of text file */

  ActualCount+1;

  if IsLast then do;

    if RecordCount=ActualCount then put "Good!";

    else do;

      put "ERROR: Count mismatch " RecordCount= ActualCount=;

      abort; /* Abort, etc - insert preferred action here */

    end;

  end;

run;

filename MyInFile clear;

Kurt_Bremser
Super User

If you don't want to go the SAS way as described above:

X "COUNT=`cat xx.dat|wc -l`;echo `expr $COUNT - 1` > xx_a.dat;cat xx.dat >> xx_a.dat;rm xx.dat;mv xx_a.dat xx.dat";

expr $COUNT - 1 is used if you don't want to count line 1 with the column names. Otherwise, you can directly do echo `cat xx.dat|wc -l` > xx_a.dat

for the second problem:

filename osmcd pipe "COUNT=`head -1 yyy.csv`;COUNTR=`cat yyy.csv|wc -l`;if [[ $COUNT -ne $COUNTR ]];then echo wrong;else echo good;fi";

data _null_;

infile oscmd;

input result $5.;

if result = 'wrong' then abort;

run;

jhaque
Calcite | Level 5

Hi KurtBremser, Thank you for your comments. I hope it will serve my purpose. The thing is paths are different for both files and it differs depends on server type so I took a macro variable which holds the common path for all and developed as given as below:

%let start_dir=/root/dev;

filename fileref  "&start_dir./room/moon/yyy.csv";

filename fileref  "&start_dir./star/sun/xx.csv";

Please guide How can I implement your code with the help of above fileref for both files?

Kurt_Bremser
Super User

Put the filename in a macro variable:

%let csvfilename=&start_dir./star/sun/xx.csv;

Then you can use that:

X "COUNT=`cat &csvfilename.|wc -l`;echo `expr $COUNT - 1` > &csvfilename._a;cat &csvfilename >> &csvfilename._A;rm &csvfilename.;mv &csvfilename._a &csvfilename.";

Note that the macro variable will be expanded by the macro processor because of the double quotes on the outside; the single quotes within the double quotes don't keep the macro processor from acting (what they usually do).

%let csvfilename=&start_dir./room/moon/yyy.csv;

filename osmcd pipe "COUNT=`head -1 &csvfilename.`;COUNTR=`cat &csvfilename.|wc -l`;if [[ $COUNT -ne $COUNTR ]];then echo wrong;else echo good;fi";

jhaque
Calcite | Level 5

Hi Thanks.

I am facing this warning msg.and not getting expected result.But once i run it in unix it gives correct result as expected.Thanks plz help.

 

WARNING: The quoted string currently being processed has become more than 262 characters long. You may have unbalanced quotation

marks.

Even I tried below code breaking long X statement.But not getting expected result.

jhaque
Calcite | Level 5

X "COUNT=`cat &csvfilename.|wc -l`;echo `expr $COUNT - 1` > &csvfilename._a";

X "cat &csvfilename >> &csvfilename._a;rm &csvfilename.";

X "mv &csvfilename._a &csvfilename.";

jhaque
Calcite | Level 5

Thanks a lot KurtBremser, I got the result..

suga badge.PNGThe SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment. 

Join SUGA 

CLI in SAS Viya

Learn how to install the SAS Viya CLI and a few commands you may find useful in this video by SAS’ Darrell Barton.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 10 replies
  • 4025 views
  • 2 likes
  • 6 in conversation