BookmarkSubscribeRSS Feed
Augusto
Obsidian | Level 7

Hi everyone,

Is there a way to check if a file is totaly transfered before start importing it?

I am transfering a file to the SAS server and I would run my infile code only when the transfer has finished.

This file has about 50 GB, i tried using this code below BUT IS NOT CORRECT BECAUSE THE FILE have_2012*.TXT is still in transfering and my table WANT WILL NOT HAVE all the records inside.

data _null_;

do until (fileexist("/INBOX/have_2012*.TXT"));

call sleep(5);

end;

run;

data want;

infile "/INBOX/have_2012*.TXT" MISSOVER DSD lrecl=32;

input

  @0001 ANO_MVM     $004.

  @0005 MES_MVM     $002.

  @0007 COD_AGN     $004.

  @0011 COD_OPR     $013.

  @0024 ACERTO_OPR  $002.

  @0026 COD_CMT_OPR $002.;

run;

thanks for any suggestion.

Augusto Souza

13 REPLIES 13
FriedEgg
SAS Employee

How is the file being transfered?

Augusto
Obsidian | Level 7

BY COPY AND PASTE, SO I CAN FOLLOW THE FILE SIZE BEING INCREASED BY WinSCP TOOL.

I AM IN WINDOWS O.S.

FriedEgg
SAS Employee

What is the server you are transfering to?  Why are you starting the SAS process before the transfer has started in the first place?

SASKiwi
PROC Star

I haven't tried it myself but how about:

rc=filename('myfile',fname);

fid=fopen('myfile');

I would imagine the file open would fail because it would still be locked while it is being transferred, particularly if you opened the file for update.

ballardw
Super User

If you system and the server are running with the same (or close enough time stamps) you might be able to use the DOPEN, DOPTNUM and DINFO functions to read time stamps of the given file until it looks like the time stamps quit updating.

Augusto
Obsidian | Level 7

Hi Guys,

Firstly, sorry for my english I am from Brazil, sencond thanks so much for the answers.

Well, It has already transfered and I ran my process thanks.

Welll, I imagined something like, to check the file size. As I mentioned before, this file has about 50 GB, I don't know if this file is locked even when its being transfered, because I can import it but not all the records. In my machine i have windows xp and the SAS is installed in unix. Through WinSCP in windows I can follow the transfer process until its finished.

I am not sure, but maybe I could check it through unix commands (any suggestion?).

if anyone knows any way to check it, It will be greatly appreciated.

Thanks

Augusto Souza.

Lulu_W
Fluorite | Level 6

I haven't tried myself, but how about using some scripts (like windows batch command) to copy and paste the file and create a dummy file (like success.txt) when the transfer process is finished, so you can check the dummy file to see if the transfer is completed and if it is there start reading; else wait for a while and repeat the process.

SASKiwi
PROC Star

Did you try the FOPEN while the file was transferring? Using the 'U' option will try to open it for update which should fail if the file is still being written.

data _null_;

rc=filename('myfile',fname);

fid=fopen('myfile','U');

put rc = fid = ;

run;

If fid = 0 then you know the open failed. Also check that rc = 0 to ensure the filename function worked.

Ksharp
Super User

or maybe can use PIPE to calcualted the size of these files after very Five minutes .

If there is no change ,then use the code to import.

Ksharp

Augusto
Obsidian | Level 7

Ksharp,

Could you post an example of using pipe?

Thanks

Ksharp
Super User

OK. Here is an example, if your site can use PIPE. I copy all the tables at a directory into WORK library.

%macro sleep_copy;
%local size pre_size;
%do %until(&pre_size=&size);
%let pre_size=&size;
filename x pipe 'dir c:\test\*.sas7bdat';
data _null_;
call sleep(200);
infile x dlm=' ' expandtabs end=last;
input date ?? : yymmdd12. time ?? : time6. size ?? : comma32. name : $40.;
total_size+size;
if last then call symputx('size',total_size);
run;
%end;
libname x v9 'c:\test';
proc  copy in=x out=work memtype=data;
run;
%mend;


%sleep_copy


Ksharp

Augusto
Obsidian | Level 7

Thanks so much Ksharp.

Augusto Souza

Augusto
Obsidian | Level 7

Thanks SASkiwi. I'll make the try.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

What is Bayesian Analysis?

Learn the difference between classical and Bayesian statistical approaches and see a few PROC examples to perform Bayesian analysis in this video.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 13 replies
  • 1940 views
  • 6 likes
  • 6 in conversation