Windows Error Code: 87 when downloading data from WRDS

[liuo1]

Hi SAS Community,

I have run into an issue when downloading a large data file after cleaning it with remote connection to WRDS. I am using SAS through Citrix Workspace and connecting to the WRDS cloud. The data file download was estimated to be over 40 hours long and 6 hours in I ran into this error. I have attached the error message below. If anyone has any information on what the issue is and how to fix it, that would be much appreciated!

Kind regards,

liuo1

[japelin]

traceout.sas7bdat.lck is the lock file used to rewrite the data set.
Since ht_normal_write returns the error ERROR_INVALID_PARAMETER for this file, I think the process of writing or rewriting the lock file has failed.

I'm assuming your PC is Windows since the destination is \\Client\C$\Users..., but I suspect that the Citrix session disconnected and failed to write locally, or that the WRDS side disconnected.

Since 6 hours is long enough, let alone 40 hours, I think it would be safer to download the desired dataset in a separate process without using SAS and then process it with SAS.
Or, if you really want to use SAS to download the data, how about dividing the target data set into 10 parts, downloading it 10 times, and then proc append at hand?

[liuo1]

Thanks for your reply! The data itself before cleaning is around 40-50gb so I am not able to open it after downloading (straight from WRDS database) given RAM limits. This is why I am attempting to clean it remotely on the WRDS cloud, however, the cleaned dataset is still very large. Perhaps I can try and divide the dataset and download each one as you suggested. Would you happen to know the code which can divide a dataset into 10 approximately same amount of observations (rows) on SAS? Then it can just be appended on top of each other once downloaded.

[japelin · Accepted solution]

try this

%Macro MDivide(in=, out=, cnt=);
  data _null_;
    dsid=open("&in",'I');
    if dsid>0 then do;
      nobs=attrn(dsid,'nlobsf');
      call symputx('nobs',nobs);
      rc=close(dsid);
      call symputx('div',ceil(nobs/&cnt));
    end;
  run;
  %do i=1 %to &cnt;
    %let fstobs=%eval((&i-1)*&div+1);
    %let getobs=%eval(&i*&div);
    data &out&i;
      set &in(firstobs=&fstobs obs=&getobs);
    run;
  %end;
%Mend;

/* example */
%MDivide(in =sashelp.baseball
         ,out=work.outds
         ,cnt=10);

[liuo1]

Thanks! I will give it a shot!