DATA Step, Macro, Functions and more

Connection Speed Testing using FTP

Reply
Super Contributor
Posts: 358

Connection Speed Testing using FTP

Hi All:

I have an interesting problem and I don't even know whether it is possible to do.

I would like to test the connection speed to a remote server and I'm thinking that the best way to do this is to FTP.  If I upload and download a file using an FTP, can I capture the statistics as to how many bytes were transfered and how long it takes?  Quick calculation would give me a general idea of the connection speed.

My reasons for this are that we are planning to FTP to a client  files as part of a project and I would like to get some idea as to how long this process will take.  There are a large number of files of different sizes.  We currently have no idea whether the process will take 10 minutes or 10 hours. 

I would like to do the FTP connection test with SAS because we will be using SAS to send and reveive the files in the future.

Thanks in advance.

Trusted Advisor
Posts: 1,301

Connection Speed Testing using FTP

If you have unix you could use lftp and dd

filename tmp temp;

%let tmp=%sysfunc(pathname(tmp));

filename tmp clear;

filename ftp_test pipe "lftp site -u user,pass -c 'cat /path/to/file.txt' | dd of=&tmp";

filename in "&tmp";

data _null_;

infile ftp_test firstobs=3;

input @;

put _infile_;

run;

data ftp_file;

  infile in;

  input yada_yada $;

run;

I beleieve the Filename statement for FTP access from SAS follows a similar pattern, ie. the file is read from FTP to disk as a temp file, then read from temp file into DATA step.  I could be wrong about this, but if so this spends an extra I/O cycles doing so but will give you the speed at which the file is read from the FTP and written to local disk.  You could also pipe the file from the FTP the same way and append the information from dd to the last three lines of the incoming file instead, just remove the of= part of the command in the pipe.

Super User
Posts: 10,023

Re: Connection Speed Testing using FTP

Why don't you use PING command + '>' to re-direct  the result into a file and see the ping value.

I think it is better way.

Ksharp

Super Contributor
Posts: 358

Re: Connection Speed Testing using FTP

Thanks to everyone who took time to think about this.

Unfortunately, I am not running Unix so I can't use any processes native to that system.

Also - a PING will not tell me the time it takes to upload or download a file.  I am already running a program that records the ping times.

From my experiments last night, here is what I came up with...

Firstly, I ran a PROC PRIINTTO to redirect the LOG file to a permanent dataset.  Then I created a macro program that executes the FILENAME myfile FTP etc. (with the DEBUG option) and reads the file.  The results from this are saved in the permanent log file. 

I can then use a macro %do statement to run the program with a SLEEP(nn) between each iteration.  I can sit back and let this program run for a while unattended.

Now I just need to write a little program to extract the data from the log file.

Ask a Question
Discussion stats
  • 3 replies
  • 641 views
  • 0 likes
  • 3 in conversation