Help using Base SAS procedures

How to check if an external file is ready to be imported?

Reply
Frequent Contributor
Posts: 80

How to check if an external file is ready to be imported?

Hi everyone,

Is there a way to check if a file is totaly transfered before start importing it?

I am transfering a file to the SAS server and I would run my infile code only when the transfer has finished.

This file has about 50 GB, i tried using this code below BUT IS NOT CORRECT BECAUSE THE FILE have_2012*.TXT is still in transfering and my table WANT WILL NOT HAVE all the records inside.

data _null_;

do until (fileexist("/INBOX/have_2012*.TXT"));

call sleep(5);

end;

run;

data want;

infile "/INBOX/have_2012*.TXT" MISSOVER DSD lrecl=32;

input

  @0001 ANO_MVM     $004.

  @0005 MES_MVM     $002.

  @0007 COD_AGN     $004.

  @0011 COD_OPR     $013.

  @0024 ACERTO_OPR  $002.

  @0026 COD_CMT_OPR $002.;

run;

thanks for any suggestion.

Augusto Souza

Trusted Advisor
Posts: 1,301

How to check if an external file is ready to be imported?

How is the file being transfered?

Frequent Contributor
Posts: 80

Re: How to check if an external file is ready to be imported?

BY COPY AND PASTE, SO I CAN FOLLOW THE FILE SIZE BEING INCREASED BY WinSCP TOOL.

I AM IN WINDOWS O.S.

Trusted Advisor
Posts: 1,301

How to check if an external file is ready to be imported?

What is the server you are transfering to?  Why are you starting the SAS process before the transfer has started in the first place?

Super User
Posts: 3,183

Re: How to check if an external file is ready to be imported?

I haven't tried it myself but how about:

rc=filename('myfile',fname);

fid=fopen('myfile');

I would imagine the file open would fail because it would still be locked while it is being transferred, particularly if you opened the file for update.

Super User
Posts: 10,876

How to check if an external file is ready to be imported?

If you system and the server are running with the same (or close enough time stamps) you might be able to use the DOPEN, DOPTNUM and DINFO functions to read time stamps of the given file until it looks like the time stamps quit updating.

Frequent Contributor
Posts: 80

How to check if an external file is ready to be imported?

Hi Guys,

Firstly, sorry for my english I am from Brazil, sencond thanks so much for the answers.

Well, It has already transfered and I ran my process thanks.

Welll, I imagined something like, to check the file size. As I mentioned before, this file has about 50 GB, I don't know if this file is locked even when its being transfered, because I can import it but not all the records. In my machine i have windows xp and the SAS is installed in unix. Through WinSCP in windows I can follow the transfer process until its finished.

I am not sure, but maybe I could check it through unix commands (any suggestion?).

if anyone knows any way to check it, It will be greatly appreciated.

Thanks

Augusto Souza.

New Contributor
Posts: 4

Re: How to check if an external file is ready to be imported?

I haven't tried myself, but how about using some scripts (like windows batch command) to copy and paste the file and create a dummy file (like success.txt) when the transfer process is finished, so you can check the dummy file to see if the transfer is completed and if it is there start reading; else wait for a while and repeat the process.

Super User
Posts: 3,183

Re: How to check if an external file is ready to be imported?

Did you try the FOPEN while the file was transferring? Using the 'U' option will try to open it for update which should fail if the file is still being written.

data _null_;

rc=filename('myfile',fname);

fid=fopen('myfile','U');

put rc = fid = ;

run;

If fid = 0 then you know the open failed. Also check that rc = 0 to ensure the filename function worked.

Super User
Posts: 9,779

Re: How to check if an external file is ready to be imported?

or maybe can use PIPE to calcualted the size of these files after very Five minutes .

If there is no change ,then use the code to import.

Ksharp

Frequent Contributor
Posts: 80

How to check if an external file is ready to be imported?

Ksharp,

Could you post an example of using pipe?

Thanks

Super User
Posts: 9,779

How to check if an external file is ready to be imported?

OK. Here is an example, if your site can use PIPE. I copy all the tables at a directory into WORK library.

%macro sleep_copy;
%local size pre_size;
%do %until(&pre_size=&size);
%let pre_size=&size;
filename x pipe 'dir c:\test\*.sas7bdat';
data _null_;
call sleep(200);
infile x dlm=' ' expandtabs end=last;
input date ?? : yymmdd12. time ?? : time6. size ?? : comma32. name : $40.;
total_size+size;
if last then call symputx('size',total_size);
run;
%end;
libname x v9 'c:\test';
proc  copy in=x out=work memtype=data;
run;
%mend;


%sleep_copy


Ksharp

Frequent Contributor
Posts: 80

How to check if an external file is ready to be imported?

Thanks so much Ksharp.

Augusto Souza

Frequent Contributor
Posts: 80

How to check if an external file is ready to be imported?

Thanks SASkiwi. I'll make the try.

Ask a Question
Discussion stats
  • 13 replies
  • 295 views
  • 6 likes
  • 6 in conversation