BookmarkSubscribeRSS Feed
AlexMar
Calcite | Level 5

The following code maps a network drive from within data step, and works properly, as below: 

  data _null_;

   test3= \\server\shared\directory\sub-directory\sub-sub-directory;

   command=' net use Z: '||test3 ;

   call system(command) ;

   run;

Exactly the same code segment in a bit more complex data step does not work, as follows:

   data _null_ ;

   set Library01.File002;
    if length(Variable003) < 40

       then do;
       command='md '||Variable003 ;
       call system(command) ;
            end;

       else do;
       test3= \\server\shared\directory\sub-directory\sub-sub-directory;

       command=' net use Z: '||test3 ;

       call system(command) ;

            end;
   run;

Any explanation? Any suggestion how to overcome this problem?  THANK YOU!

7 REPLIES 7
Ksharp
Super User

Why would you let SAS do OS task ?

Why not let SAS make a BAT file or a Shell script and run it at OS command line ?

Kurt_Bremser
Super User

It is always good practice to explicitly specify lengths for character variables, as the first (by SAS) assumed length while compiling the data step may be insufficent later during execution. You also did not specify what exactly did not work (SAS error message, OS error message, ...).

jakarman
Barite | Level 11

Kurt, I am amazed would expect you to direct to the OS level for those things.

---->-- ja karman --<-----
Tom
Super User Tom
Super User

The biggest differences between the two steps are:

1) In the second you first create COMMAND variable by concatenating and constant and a variable that must have come from the input data step.  SAS will probably set the length of this COMMAND variable to be three bytes longer than the old VARIABLE003 variable. So it probably is not long enough to later hold the full NET USE command.

2) The input data set could have many observations. Why are you trying to map all of them to the same drive letter?

3) Why are you mapping paths to drive letters anyway?  Why not just use the UNC path and avoid the drive letter completely.

You should use a PIPE to run OS commands, that way you can retrieve the messages that the OS sends back to you.

data _null_ ;

  length command $300 ;

  set Library01.File002;

  if length(Variable003) < 40 then

    command=cats(' ','md',Variable003)

  ;

  else

    command=cats(' ','net use Z:'

         ,'\\server\shared\directory\sub-directory\sub-sub-directory')

  ;

  infile cmd pipe filevar=command end=eof;

  do while (not eof);

    input ;

    put _infile_;

  end;

  stop;

run;

AlexMar
Calcite | Level 5

Gentlemen, thank you for responding and advice. I will try some of the suggestions, especially detailed ones provided by Tom, on Monday when I will have access to SAS and data.

Tom, the task for which I am trying to use this code is of course more complicated than the example I used to illustrate it. Namely, prior to migrating some half a million files of various nature (type, size, etc.) from one directory structure to another, files were reviewed by respective users and categorized in 4 groups (delete, long tail, important, critical). This information is found only in an Excel file, each row including file name, current/future location (directory path) and category.  The task is using the information from the Excel file to delete or move files to their new locations as per respective categories. The path to some of the current locations exceeds the maximum of 259 characters (Naming Files, Paths, and Namespaces (Windows)), and respective files can't even be accessed there via Windows. So the idea of using SAS was:

(1) read file name, path and assigned category (delete, long tail, etc.) from the already imported Excel file;

(2a) if the path name is shorter than the maximum allowed path name length, then delete or move the file from the old to the new location (using "call system (....)");

(2b) if the path name is longer than the maximum allowed path length, first brake the path string into, say, two parts, then use the first part to map to a drive (using "call system ('net use Z:'first part'), and only then using as path the second part (Z:\second part) delete or move the respective file.  Because the paths exceeding the maximum length are different, I cannot 'map drive' only once prior to running any SAS code, but have to do this for each specific path, and after completing the 'delete' or 'move' to free the drive (net use Z: /delete) to be available for the next path exceeding the max length.

I believe, I managed to this with macro loops (at least with test files), but it worked very slow, estimating over 24 hours for the whole job, if not considerably more taking into account the large size of some files. This is the reason I am trying to do this in one data step.

There might be, of course, other approaches.  One I explored and did not work on our system (unless, I didn't do it properly) was "To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".

Note  The maximum path of 32,767 characters is approximate, because the "\\?\" prefix may be expanded to a longer string by the system at run time, and this expansion applies to the total length." (Naming Files, Paths, and Namespaces (Windows))                               

Thank you, again, and again - any other suggestions/advice in light of the above would be greatly appreciated. / Alex M.

jakarman
Barite | Level 11

Alex of course that will be slow as you are causing a lot of IO may be even network related. That can become easily a trashing one. Trashing is the system will become very busy but hardly doing any real work as it mostly trying correcting interrupts for doing work. Imagine you have 20 managers continuously asking for how the work proceeds but giving no room to do any work.  The problem is not the coding but understanding what is happening by that code.

Advices:

- try to avoid copying. Moving is much faster. Moving and Replacing Files (Windows) there is a move command.  https://technet.microsoft.com/en-us/library/cc772528.aspx

  The difference is that is not copying data but instead will try to move the pointer to the data in the filesystem so it will appear within another directory. (renaming)

- run the commands in xsync (symchronous mode) to prevent that too many processes are started running at the same time. You could create some lists that could run in parallel (3-4 max)  and start those separately.

- those very long names should be no problem using unc names. No need the split those and using drive letters.

- Assuming you have sas-datasets than use the sas "proc copy" with move option.

- having libraries to new non-existence directories use: "options dlcreatedir"  ;   
  New directories will be createn as soon being used with those libnames.

- Yes, you can mix drectories with different types of data sas specific and non-sas types (eg pdf word text)  at Windows/Unix (no not classic mainframe) . That is often more complicating than simplifying the job. Each SAS version is having a unique structure an version types may not be mixed.

Think on placing the SAS specific types in segregated directories. 

---->-- ja karman --<-----

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 7 replies
  • 5282 views
  • 1 like
  • 5 in conversation