Hi everyone !
i've tried many times to copy a csv file to a hadoop cluster.
The datafile is 4.6 Go.
Randomly, the file is copied entirely or stops at 2.1 Go.. 1.5Go..1.6Go sometimes
i dont know how to make sure that the file is correcly copied, there is no error on the log.
Did you ever had that kind of problem ?
proc hadoop username="&user." password="&mdp." verbose;
hdfs delete="&chemin_hdfs./&fichierIn." nowarn;
run;
proc hadoop username="&user." password="&mdp." verbose;
hdfs copyfromlocal="&chemin_sorties.\&fichierIn." out="&chemin_hdfs." ;
run;