X cmd in a DI Studio job

[cnilsen]

Hello,

My headache for today seems simple in theory, but I cannot get it to work:

At the end of a DI Studio job (v 4.901) , in the postcode of a transformation, I want to execute an X cmd to delete tables in my archive directory that are over 7 days old. Sounds easy enough right? 

in this transformation, I am already using an X cmd to copy the file I just processed into an archive folder based on the status of the job being 'successful'. This works with no problems whatsoever:

X cp -a /dmadata/AS400/CDH_* /dmadata/AS400_ARCH/;

Now after this, I also want to delete any tables in this same archive directory that are older than 7 days.

I have tried several variations of :

x find /dmadata/AS400_ARCH/CDH* -mtime +7 -exec rm {} \;  

the job runs without error, but doesnt do anything. 

I have tried to use this code without the mtime check, as well as with a fully defined table name.. but nothing works.

I tried this version:

x rm /dmadata/AS400_ARCH/CDH* ;

and everything within the directory was deleted..  so what's the issue?  

is there another piece I'm missing, or a format change that could work.? (and yes, I even moved this process to the front of my 'job' to see if it makes a difference, and the results were the same)

I'm stuck and looking for someone who might have had this same issue before..

Thanks,

[Patrick]

@cnilsen

That's definitely Unix/Linux scripting and not a SAS question.

x find /dmadata/AS400_ARCH/CDH* -mtime +7 -exec rm {} \;

This command looks o.k. to me and I can find similar examples on the internet. I'd probably add "-type f" to your command so that you don't delete sub-directories by mistake.

The only explanation I'm having: There are no files older than 7 days in your archive directory. Have you checked?

If you want to preserve the "mtime" when copying files to the archive then I believe you need also to use the "-p" switch as part of your cp command.

[cnilsen]

Hi Patrick,

Everything I have looked at online, or with the help of a coworker shows it as a valid unix command. and, if I execute it in that environment.. it works perfectly. I dont understand why DI is having a hard time with it in this particular format. I have tried with and withiout the file type qualifier, as well as the date test. No luck either way. 

Perhaps an issue with the -Exec or Find command?  Becuase of the fact I am changing to a unix command, SAS doesnt log anything about this in the log. 

When I am doing my move/copy I am using the '-a' so that I can retain the original attributes of the table. 

I figure I cant possibly be the only person to have had this issue... 

[LinusH]

If you want feed back, don't use X. If this is SAS tables, using PROC DATASETS seem more appropriate.

[cnilsen]

LinusH-

These are not SAS tables. they are actually delimited files on a unix server that I process with DI Studio and into a SAS table. I am able to execute the X cmd to move the delimited tables into an archive directory also on the unix box. I was also hoping to use X to delete older tables within that same archive directory as part of the same job.

[Patrick]

@cnilsen

I've done similar things than what you're trying to do and they work out of SAS.

Let's first get DIS out of the way. You're using user written code and DIS is here nothing else than the client which you use to send code to the SAS server for execution. The code as such won't get changed by DIS and you could run this code also out of another client (ie. EG or SAS Studio) which connects to the same environment.

On the SAS server side when using an X command: This command should in principle just get passed to the host for execution.You might have to add some quoting and masking if you use characters like & which are SAS Macro tokens.

If you want to capture the feedback from the OS for your command then don't use an X command but a filename pipe as explained here:

http://blogs.sas.com/content/sgf/2016/03/11/using-a-pipe-to-return-the-output-of-an-operating-system...

Thanks to @JuanS_OCS who has pointed me into the right direction using a pipe in his answer to this question: 

https://communities.sas.com/t5/Administration-and-Deployment/sas-recover-metadata-executed-via-call-...

[cnilsen]

Patrick,

Thanks for the help. I'll read up on the articles you sent links to and make some changes in my program. Hopefully this will give me the needed feedback to see where things are going wrong, or it will be successful finally.

Regards.

[Tom]

You haven't quoted the string that you are passing to the X command, so that can cause errors.

For example in this one the semi-colon is part of the FIND command, but SAS will instead take it as marking the end of the X command which is probably what is causing your command not to run.

x find /dmadata/AS400_ARCH/CDH* -mtime +7 -exec rm {} \;  

You could change from using \; to + in the find command. A change that you probably want any way.

x find /dmadata/AS400_ARCH/CDH* -mtime +7 -exec rm {} +;

You could add the quotes.

x 'find /dmadata/AS400_ARCH/CDH* -mtime +7 -exec rm {} +';

What I like to do is use a DATA step to run commands. That way you can read the output. For example you could tell the FIND command to list the files that it found befor it removes them. 

data _null_;
  infile 'find /dmadata/AS400_ARCH/CDH* -mtime +7 -ls -exec rm {} +' pipe;
  input;
  put _infile_;
run;

[cnilsen · Accepted solution]

Thanks for the idea Tom,
I'm relatively new to SAS so this is an option I had known about. I'll give it a try and see what happens.

I appreciate you using my code to do the data step.. I can more easily see how the process works.

Regards.