BookmarkSubscribeRSS Feed
DavidHu
Calcite | Level 5

Hi There,

I am new to SAS. I am assigned a task to automate some manuel SAS job. I am planning to do it in java. In the processes, I need to call different ".sas" script files based on the different conditions and dynamic parameters need to be passed to the ".sas" script.

Is there any ways can do it?

Thanks a lot in advance!

David

15 REPLIES 15
AhmedAl_Attar
Rhodochrosite | Level 12

David,

In order to get a suitable answer, you'll need to specify

- Where does your Java Client App run? Windows or Unix?

- Do you want to run SAS locally on the same client machine or on a remote machine?

Ahmed

DavidHu
Calcite | Level 5

Thanks a lot Ahmed!

At first I planed to installed the Java App on the SAS Server which is running on a Windows 2003 server. But after I talked with the SAS Server Admin, I was told I can only run the java App on a remote client workstation which is a Windows OS too.

The .sas scripts and files used by the scripts are in the server too but are all mapped to the client workstation as a network driver. Currently, a user will find the new file and select a script based on the new file name, manually change the path and file name to reflect the real path and file name to be parsed, save the script, and then run the script by EG 4.5.

What I need to do is call different sas scripts based on different files need to be parsed by the .sas scripts. For example, if the file name is abc.txt, I will call abc_script.sas to do the parse. In abc_script.sas, I need to change the file path and name in the "abc_script.sas"  to ..\..\abc.txt so that the script to parse the correct file, and then create a report called "abc,xls". So file name and path must be passed to the script correctly.

How Can I do that?

Thanks a lot!

David

AhmedAl_Attar
Rhodochrosite | Level 12

OK, on Windows you can use the following command to start a batch SAS job

c:\program files\SAS\SASFoundation\9.2\sas.exe

  -sysin c:\mysas\programs\prog1.sas

  -config c:\program files\SAS\SASFoundation\9.2\sasv9.cfg

You may need to change 'c:\program files\SAS\SASFoundation\9.2' to match your SAS Server settings.

You should replace 'c:\mysas\programs\prog1.sas' with your own SAS scripts location.

Ahmed

DavidHu
Calcite | Level 5

Hi Ahmed,

I cannot use sas.exe because I was told we do not have this in our client.

At first I think the app will be very simple because I just need to do a native calls like you said

sas -sysparm 'file_path_and_name' sas_script.sas


But I am told I cannot use above call because we do not have the above command available.


How can I have the above command available in the SAS client? I searched the web, EG, SAS Connect, and so no do not have this command available.


Is the SASFoundation a client I need to install to make the sas.exe available? where and how I can get it if the answer is 'yes"?


Thanks a lot Ahmed.


David

AhmedAl_Attar
Rhodochrosite | Level 12

If you have SAS Enterprise Guide (EG) then you can use the %include program.sas from within EG.

The way EG works, it runs locally on your client machine, but it submits the SAS statement(s) on the SAS Server it connected to.

DavidHu
Calcite | Level 5

Hi Ahmed,

Yes we are using EG 4.5 as client.

The current processes is like that: there are file copied from a remote file server to SAS server continuously. Periodically (half hour, 1 hour) a user is coming to check if there are any new files copied by using EG. If there is a new one, the user is going to find correct sas script based on the file name. Then he/she will open the selected sas scripts, modify the file name and path to the real path, file name, submission fiscal year. Then he/she will saved the script, and run the script in EG.

This is tiresome because there are so many files. So I was assigned to automate this processes.

I will have a Java application always running to check if there is a new file. If there is a new file, the app will select the right scripts based on the file name. Instead of harcoded file path, name and fiscal year, I will change these 3 into a dynamic parameter in the script. Then I will call the script by using a native command call in Java:

sas -sysparm 'file_path_and_name' sas_script.sas


This is my plan. But EG does not have sas.exe available for the Java app to call.


I want to know what else I can do?


Thanks a lot!

David

AhmedAl_Attar
Rhodochrosite | Level 12

OK, now things are a bit clear.

You don't need to complicate your life by using Java!!!

All you need is a SAS program that can read the arrival of these new file(s) and apply the required changes. Once you have developed that SAS program, all you need is scheduling to run on a periodic interval using the AT scheduler on your Windows Server.

Here are few relative White papers to look at for inspiration Smiley Wink

Ahmed

DavidHu
Calcite | Level 5

Thank you very much Ahmed. I selected java first because My strength is Java and I though I could use the command line command

sas -sysparm 'file_path_and_name' sas_script.sas


to run the SAS scripts.


The above links are all about the job scheduling. I read first one and the 3rd one before, and I will read the rest of them. 


Is it easy to use SAS scripts to do the file system scan to find new files, to compare the time stamps of the file, to create a bunch of sub folders and to copy files?


Thanks!


David

AhmedAl_Attar
Rhodochrosite | Level 12

Hi David,

Yes, it's easy to use SAS for external file system processing.

Check the following SAS 9.2 On-line help documentation for Functions and Call Routines, You'll need to scroll down to the "External Files" Category. Most of the functions documentation would have usage examples, that you can utilize to get what you are looking for.

Here are few additional links to illustrate more usage examples

DavidHu
Calcite | Level 5

They are very useful and helpful to me! Thank you very much Ahmed!

David

DavidHu
Calcite | Level 5

Thank you very much Ahmed!

I spent some time on SAS and write a integration sas script. Can you help me to take a look? I think I may have errors in the folder recursive searching for the subfolders and files.

Thank you very much for your help!

Davy

intervalForFileValidation = 1; /*** in hours ***/

currentDate = Today();

currentYear = Year(currentDate);

currentYearIn2Digits = substr(currentYear,3,2);

endyearIn2Digits = currentYearIn2Digits + 1;

currentSubmissionPeriod = "" + currentYearIn2Digits + "" + endyearIn2Digits;

/** root for project: c:\SAS\data\users\DDM\OCCI\OCCI_AuditYYYY ****/

constantPath = "c:\SAS\data\users\DDM\OCCI\OCCI_Audit";

%Let mypath = cats(of constantPath currentSubmissionPeriod);

rawDataFilePath = &mypath.\Data\Rawdata

data rawDataDir;

  length filename dir fullpath $256;

  dir = "&dataFilePath";

  rc = filename("readdir","&rawDataFilePath");

  did = dopen("readdir");

  memcount = dnum(did);

  /***** 1. Scan the given file system of rawDataFilePath folder, subfolder and files **************************************************/

  do i = 1 to memcount;

  filename = dread(did,i);

  fullpath = trim(dir) !! "\" !! filename;

  fid=fopen("readdir");

  fileCreateDate=finfo(fid,"Create Time");

    close=fclose(fid);

  timePassed = (Today() - fileCreateDate)/InHours();

  /***** 2. If the file is a TXT one and if it has not been validated before (file create date < 1 hour), populate it *********/

  do while(timePassed > intervalForFileValidation and index(filename,"TXT") > 1);

  if(index(filename,"AIP") > 1) then

  %Let mvType   = AIP;

  if(index(filename,"AMB") > 1) then

  %Let mvType   = AMB;

  if(index(filename,"AMH") > 1) then

  %Let mvType   = AMH;

  if(index(filename,"CCC") > 1) then

  %Let mvType   = CCC;

  if(index(filename,"RHB") > 1) then

  %Let mvType   = RHB;

  if(index(filename,"CAC") > 1) then

  %Let mvType   = CAC;

  %Let mvMileStone= YE;

  %LET OCDM = Yes;

  %Let filename = &fullpath;

  %Let root = &mypath;

  %INCLUDE "&root.\Program\Shared\Initiate.sas";

  ODS path RESET;

  %INCLUDE "&publicpath\Import.sas";

  %INCLUDE "&publicpath\Match.sas";

  %INCLUDE "&publicpath\Audit.sas";

  %INCLUDE "&publicpath\Export.sas";

  OPTIONS MPrint NoSymbolGen NoXWait NoxSync;

  %GetSettings(0);

  %GoAuditing();

  ODS path RESET;

  LibName metdata Clear;

  LibName appdata  Clear;

  LibName result Clear;

  LibName keydata  Clear;

  end;

  end;

run;

rc=dclose(did);

drop rc fid did memcount i;

run;

AhmedAl_Attar
Rhodochrosite | Level 12

Davy,

The code below will generate a data set containing the fully qualified (full path) file names that meets your search criteria (Created < 1 hour AND of Type txt). It's parameter driven, so you can use it with different condition values.

Once you have the list of files data set, you can read it and perform your desired processing.

Hope this helps,

Ahmed

/* Input parameters : rootpath and maximal size of members (can be increased) */

%GLOBAL    g_constantPath

        g_currYearIn2Digit

        g_currentSubmissionPeriod

        g_rawDataFilePath

        g_array_size

        g_searchFileType

        g_intervalForFileValidation

        ;

%let g_constantPath              = c:\SAS\data\users\DDM\OCCI\OCCI_Audit;

%let g_currYearIn2Digit          = %sysfunc(putn(%sysfunc(date()),year2.));

%let g_currentSubmissionPeriod   = &g_currYearIn2Digit.%eval(&g_currYearIn2Digit+1);

%let g_rawDataFilePath           = &g_constantPath.&g_currentSubmissionPeriod.\Data\Rawdata;

%LET g_array_size                = 9999; /* maximal size of members (can be increased) */

%LET g_searchFileType            = txt;  /* Type of file to search for */

%LET g_intervalForFileValidation = 1;    /* in hours */

/* ----------------------------------------------------- */

/* Naming convention : functions and routines compiled   */

/* by proc FCMP are prefixed with 'x' to single them out */

/* ----------------------------------------------------- */

PROC FCMP OUTLIB=work.funcs.dir;

    /* -------------------------------------- */

    /* New ancillary function : open an entry */

    /* -------------------------------------- */

    FUNCTION xdiropen(dir $);

        LENGTH dir $ 256 fref $ 8; *maximal size for filepath is set tup to 256, can be increased;

        /* a fileref is assigned to the incoming path */

        rc = FILENAME(fref, dir);

        /* path can be accessed ...*/

        IF rc = 0 THEN

        DO;

            /* .. then create a pointer with DOPEN */

            did = DOPEN(fref);

            /* and free up fileref */

            rc = FILENAME(fref);

        END;

         /*... else catch exception for inaccessibility */

        ELSE

        DO;

            msg = SYSMSG();

            PUT msg '(XDIROPEN(' dir= ')';

            /* return code gets a missing value */

            DID = .;

        END;

        RETURN(did);

    ENDSUB;

    /* ------------------------------------- */

    /* Routine XDIRCLOSE : closing the entry */

    /* ------------------------------------- */

    SUBROUTINE xdirclose(did);

        OUTARGS did;

        rc = DCLOSE(did);

        did = .;

    ENDSUB;

    /* --------------------------------------------- */

    /* Main Routine XDIR_ENTRIES : recursive listing */

    /* input : rootpath (char string)                */

    /* output : members found below (array character)*/

    /* + number of members found + binary Flag       */

     /* --------------------------------------------- */

    SUBROUTINE xdir_entries(dir $, files

  • $, n, trunc);
  •         OUTARGS files, n, trunc;

            LENGTH dir entry $ 256 infoname $60 infoval $256;

            /* if flag ("trunc") equals 1 then stop - used for array resizing (might be unnecessary with 9.3+) */

            IF trunc THEN

                RETURN;

            /* Entry opened */

            did = XDIROPEN(dir);

            /* if issue accessing the foldern then exit with error */

            IF did <= 0 THEN

                RETURN;

            /* Browsing through every entry found */

            dnum = DNUM(did);

            DO i = 1 TO dnum;

                /* Test : is this entry a mere file ? */

                entry = DREAD(did, i);

                fid = MOPEN(did, entry);

                entry = TRIM(dir) || '\' || entry;

                /* A - Yes, entry is a file */

                IF fid > 0 THEN

                    DO;

                        /* Extract File's Create Time Information */

                        infoval=finfo(fid,'Create Time');

                        /* Calculate the age of the file in hours */

                        timePassed=intck('HOUR',input(infoval,datetime18.),datetime());

                        put 'entry=' entry 'Create Time=' infoval 'timePassed in hours=' timePassed;

                        rc = FCLOSE(fid);

                        /* if array dimension is sufficient then we store the pathname */

                        IF n < DIM(files) THEN

                        DO;

                            trunc = 0;

                            /* Only keep entries of the specified file type */

                            /* and has not been validated before            */

                            IF ((index(entry,"&g_searchFileType") > 0) AND

                                (timePassed < &g_intervalForFileValidation))THEN

                            DO;

                                n = n + 1;

                                files = entry;

                            END;

                        END;

                         /* ... else we increment Flag */

                        ELSE

                        DO;

                            trunc = 1;

                            CALL xdirclose(did);

                            RETURN;

                        END;

                    END;

                ELSE  /* B - Entry is not a file but a folder : iterate upon this new root folder */

                CALL xdir_entries(entry, files, n, trunc);

            END;

            CALL xdirclose(did);

            RETURN;

        ENDSUB;

    QUIT;

    OPTIONS CMPLIB=work.funcs;

    /* The table will store the members listed */

    DATA File_List(KEEP=file);

        /* aFiles array assignment ('a' for 'array')*/

        ARRAY afiles[&array_size] $ 256 _TEMPORARY_;

        dnum = 0;

        trunc = 0;

           /* recursive browsing through the rootpath and store members found into "afiles" */

        CALL xdir_entries("&g_rawDataFilePath", afiles, dnum, trunc);

        IF trunc THEN

            PUT 'ERROR: Not enough result array entries. Increase array size.';

        /* once array is returned, we write an observation for each value */

        DO i = 1 TO dnum;

            file=afiles;

            OUTPUT;

        END;

    RUN;

    DavidHu
    Calcite | Level 5

    Thank you so much Ahmed!

    I'm going to try modify my code use your code right now!

    DavidHu
    Calcite | Level 5

    I just have my SAS client setup and combined the above code with my brief business logic. It works perfectly!

    Thank you soooooo much Ahmed!!!!!!!!!!!!

    sas-innovate-2024.png

    Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

    Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

     

    Register now!

    How to Concatenate Values

    Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

    Find more tutorials on the SAS Users YouTube channel.

    Click image to register for webinarClick image to register for webinar

    Classroom Training Available!

    Select SAS Training centers are offering in-person courses. View upcoming courses for:

    View all other training opportunities.

    Discussion stats
    • 15 replies
    • 3134 views
    • 6 likes
    • 2 in conversation