FTP transfer issues -corrupt files

Reply
Frequent Contributor
Frequent Contributor
Posts: 133

FTP transfer issues -corrupt files

So I have a FTP script that takes a file from my home folder on one linux server to  another.

This file is used as a credential file for SAMBA. This will be used as part of a larger program for other developers to shortcut having to manually ftp  this file to different servers. 

 

When the file transfers through FTP it gets corrupted.  This is verified by manually downloading the file and reloading it through a FTP client (reflections FTP) 

 

This is the second time this has happened in that the same thing happened to excel files even when the binary option is used. 

Anyone know why this might occur-   

 

Here is the script-It works just fine in terms of moving the data. 

 

 

filename linhome "P:\cs.txt";
filename srvrdest ftp "/home/&sysuserid/cs.txt"
user=&sysuserid host="&serv" cd="/home/&sysuserid" pass="&RITOPASS" binary debug;
data _null_;
infile linhome;
file srvrdest;
input;
put _infile_;
run;
filename linhome clear;
filename srvrdest clear;

 

 

 

Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

If I run this-

 
filename COPY PIPE "scp /home/bxxx/cs.txt bxxx@xxxx8x.ox.kx.org:/home/bxxx/cs.txt";
data _null_ ;
infile COPY;
input ;
put _infile_;
run;

 

I get this error- 


(gnome-ssh-askpass:72206): Gtk-WARNING **: cannot open display: localhost:10.0
Host key verification failed.
lost connection

Esteemed Advisor
Posts: 6,723

Re: FTP transfer issues -corrupt files

scp will ask for credentials if you do not have public/private key authentication enabled. Trying to prompt for a password probably causes your error.


LB wrote:

If I run this-

 
filename COPY PIPE "scp /home/bxxx/cs.txt bxxx@xxxx8x.ox.kx.org:/home/bxxx/cs.txt";
data _null_ ;
infile COPY;
input ;
put _infile_;
run;

 

I get this error- 


(gnome-ssh-askpass:72206): Gtk-WARNING **: cannot open display: localhost:10.0
Host key verification failed.
lost connection


 

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Esteemed Advisor
Posts: 6,723

Re: FTP transfer issues -corrupt files


LB wrote:

So I have a FTP script that takes a file from my home folder on one linux server to  another.

This file is used as a credential file for SAMBA. This will be used as part of a larger program for other developers to shortcut having to manually ftp  this file to different servers. 

 

When the file transfers through FTP it gets corrupted.  This is verified by manually downloading the file and reloading it through a FTP client (reflections FTP) 

 

This is the second time this has happened in that the same thing happened to excel files even when the binary option is used. 

Anyone know why this might occur-   

 

Here is the script-It works just fine in terms of moving the data. 

 

 

filename linhome "P:\cs.txt";
filename srvrdest ftp "/home/&sysuserid/cs.txt"
user=&sysuserid host="&serv" cd="/home/&sysuserid" pass="&RITOPASS" binary debug;
data _null_;
infile linhome;
file srvrdest;
input;
put _infile_;
run;
filename linhome clear;
filename srvrdest clear;

 

 

 


Could it be that the binary option prevents conversion of CRLF to LF?

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

That I don't know. 

But FTP is still being problematic- 

 

I am trying scp w/o much success. 

 

Lawrence 

 

 

Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

[ Edited ]

Is there is a SCP solution-

 

I have tried different variants here- 


filename COPY PIPE "scp file='/home/b335489/cs.txt' todir='xxx9@xxx.xx.x.org:/home/b335489/'" ;
data _null_ ;
infile COPY;
input ;
put _infile_;
run;

AND

 

filename COPY PIPE "scp file='/home/b335489/cs.txt' todir='xx: password@xxx.xx.x.org:/home/b335489/'" ;

 

The first fails do to authentication and the second fails as it cannot resolve the host.   

Esteemed Advisor
Posts: 6,723

Re: FTP transfer issues -corrupt files

When using SSH (the protocol used by scp) you need to set up public/private key authentication to prevent being prompted for passwords. Once the scp command works from a commandline, you can start using it from SAS.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

IS there a way to actually input the password or do I need to set up the SSH keys? 

This is being set up in a macro so that the user inserts their password in a macro that is carried through-It is run once to copy a file-all to use SAMBA. I would just FTP the file-but it, like my excel files gets corrupted in transit.

I don't think setting up the public private keys will work in this case. 

 

Thanks 

 

Lawrence 

 

 

 

 

Esteemed Advisor
Posts: 6,723

Re: FTP transfer issues -corrupt files

Software with a respect for security will NOT let you supply passwords on the commandline or other means that can be sniffed (a simple ps -ef would reveal the password). So set up public/private keys for SSH.

 

And could you please show an example how a file is corrupted by FTP?

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

Good to know- It makes sense to avoid what could be an inherent error on a users part-   

For the excel files-it just gives an error that the file is corrupted and cannot be opened.  

 can't replicate it per se as it takes too much time to do so. 

 

But the text file is different- 

I have a simple script that outputs a file on the local unix home folder with the credentials for SAMBA 

 

data _null_;
file "/home/&sysuserid/cs.txt";

put "username = &SYSUSERID";
put "password = &WINPASS";
put "domain = CS";

run;

 

The FTP script that does work ports it to the other server-when I download  the file it looks the same-however something happens in transit. If I run the program through the management console as a batch job on the first server-the SAMBA credential file works. 

However when I attempt to use it on the second server with the same process it fails. 

 

I know the FTP is the issue as if I copy the file down through a FTP client and then reload it into the second, the SAMBA credential file now works.  It is an ascii file so I really don't understand the issue. 

 

Thanks for your help.  I know SAS well but not Unix/linux.  

 

Lawrence 

 

 

 

Esteemed Advisor
Posts: 6,723

Re: FTP transfer issues -corrupt files

More advanced text editors (ie notepad++) will tell you the text file format (DOS or UNIX) in the status line.

Or allow you to view the file in hexadecimal mode, so you can find differences that are not visible otherwise.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Frequent Contributor
Posts: 133

Re: FTP transfer issues -corrupt files

Ok-thanks. 

 

Lawrence 

 

 

Ask a Question
Discussion stats
  • 11 replies
  • 184 views
  • 1 like
  • 2 in conversation