Map network drive from within Data step

Reply
New Contributor
Posts: 2

Map network drive from within Data step

The following code maps a network drive from within data step, and works properly, as below: 

  data _null_;

   test3= \\server\shared\directory\sub-directory\sub-sub-directory;

   command=' net use Z: '||test3 ;

   call system(command) ;

   run;

Exactly the same code segment in a bit more complex data step does not work, as follows:

   data _null_ ;

   set Library01.File002;
    if length(Variable003) < 40

       then do;
       command='md '||Variable003 ;
       call system(command) ;
            end;

       else do;
       test3= \\server\shared\directory\sub-directory\sub-sub-directory;

       command=' net use Z: '||test3 ;

       call system(command) ;

            end;
   run;

Any explanation? Any suggestion how to overcome this problem?  THANK YOU!

Grand Advisor
Posts: 9,584

Re: Map network drive from within Data step

Why would you let SAS do OS task ?

Why not let SAS make a BAT file or a Shell script and run it at OS command line ?

Esteemed Advisor
Posts: 6,685

Re: Map network drive from within Data step

It is always good practice to explicitly specify lengths for character variables, as the first (by SAS) assumed length while compiling the data step may be insufficent later during execution. You also did not specify what exactly did not work (SAS error message, OS error message, ...).

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Valued Guide
Posts: 3,206

Re: Map network drive from within Data step

Kurt, I am amazed would expect you to direct to the OS level for those things.

---->-- ja karman --<-----
Esteemed Advisor
Posts: 6,685

Re: Map network drive from within Data step

Well, the usage of character variables in the data step was faulty as such, which I tried to point out.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Super User
Super User
Posts: 6,364

Re: Map network drive from within Data step

The biggest differences between the two steps are:

1) In the second you first create COMMAND variable by concatenating and constant and a variable that must have come from the input data step.  SAS will probably set the length of this COMMAND variable to be three bytes longer than the old VARIABLE003 variable. So it probably is not long enough to later hold the full NET USE command.

2) The input data set could have many observations. Why are you trying to map all of them to the same drive letter?

3) Why are you mapping paths to drive letters anyway?  Why not just use the UNC path and avoid the drive letter completely.

You should use a PIPE to run OS commands, that way you can retrieve the messages that the OS sends back to you.

data _null_ ;

  length command $300 ;

  set Library01.File002;

  if length(Variable003) < 40 then

    command=cats(' ','md',Variable003)

  ;

  else

    command=cats(' ','net use Z:'

         ,'\\server\shared\directory\sub-directory\sub-sub-directory')

  ;

  infile cmd pipe filevar=command end=eof;

  do while (not eof);

    input ;

    put _infile_;

  end;

  stop;

run;

New Contributor
Posts: 2

Re: Map network drive from within Data step

Gentlemen, thank you for responding and advice. I will try some of the suggestions, especially detailed ones provided by Tom, on Monday when I will have access to SAS and data.

Tom, the task for which I am trying to use this code is of course more complicated than the example I used to illustrate it. Namely, prior to migrating some half a million files of various nature (type, size, etc.) from one directory structure to another, files were reviewed by respective users and categorized in 4 groups (delete, long tail, important, critical). This information is found only in an Excel file, each row including file name, current/future location (directory path) and category.  The task is using the information from the Excel file to delete or move files to their new locations as per respective categories. The path to some of the current locations exceeds the maximum of 259 characters (Naming Files, Paths, and Namespaces (Windows)), and respective files can't even be accessed there via Windows. So the idea of using SAS was:

(1) read file name, path and assigned category (delete, long tail, etc.) from the already imported Excel file;

(2a) if the path name is shorter than the maximum allowed path name length, then delete or move the file from the old to the new location (using "call system (....)");

(2b) if the path name is longer than the maximum allowed path length, first brake the path string into, say, two parts, then use the first part to map to a drive (using "call system ('net use Z:'first part'), and only then using as path the second part (Z:\second part) delete or move the respective file.  Because the paths exceeding the maximum length are different, I cannot 'map drive' only once prior to running any SAS code, but have to do this for each specific path, and after completing the 'delete' or 'move' to free the drive (net use Z: /delete) to be available for the next path exceeding the max length.

I believe, I managed to this with macro loops (at least with test files), but it worked very slow, estimating over 24 hours for the whole job, if not considerably more taking into account the large size of some files. This is the reason I am trying to do this in one data step.

There might be, of course, other approaches.  One I explored and did not work on our system (unless, I didn't do it properly) was "To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".

Note  The maximum path of 32,767 characters is approximate, because the "\\?\" prefix may be expanded to a longer string by the system at run time, and this expansion applies to the total length." (Naming Files, Paths, and Namespaces (Windows))                               

Thank you, again, and again - any other suggestions/advice in light of the above would be greatly appreciated. / Alex M.

Valued Guide
Posts: 3,206

Re: Map network drive from within Data step

Alex of course that will be slow as you are causing a lot of IO may be even network related. That can become easily a trashing one. Trashing is the system will become very busy but hardly doing any real work as it mostly trying correcting interrupts for doing work. Imagine you have 20 managers continuously asking for how the work proceeds but giving no room to do any work.  The problem is not the coding but understanding what is happening by that code.

Advices:

- try to avoid copying. Moving is much faster. Moving and Replacing Files (Windows) there is a move command.  https://technet.microsoft.com/en-us/library/cc772528.aspx

  The difference is that is not copying data but instead will try to move the pointer to the data in the filesystem so it will appear within another directory. (renaming)

- run the commands in xsync (symchronous mode) to prevent that too many processes are started running at the same time. You could create some lists that could run in parallel (3-4 max)  and start those separately.

- those very long names should be no problem using unc names. No need the split those and using drive letters.

- Assuming you have sas-datasets than use the sas "proc copy" with move option.

- having libraries to new non-existence directories use: "options dlcreatedir"  ;   
  New directories will be createn as soon being used with those libnames.

- Yes, you can mix drectories with different types of data sas specific and non-sas types (eg pdf word text)  at Windows/Unix (no not classic mainframe) . That is often more complicating than simplifying the job. Each SAS version is having a unique structure an version types may not be mixed.

Think on placing the SAS specific types in segregated directories. 

---->-- ja karman --<-----
Ask a Question
Discussion stats
  • 7 replies
  • 1035 views
  • 1 like
  • 5 in conversation