Is the following an efficient way of running several sas programs in parallel? Each program uses the same data and filters it by year 2006,2007...ect. and runs independent regression like anslysis on them. The outputs are eported as CSV files with year indexed names. #!/bin/bash
#SBATCH -t 4:00:00
#SBATCH --job-name=SAS01
#SBATCH -N 1
#SBATCH -n 16
#SBATCH --partition=Bigg
. /etc/profile.d/modules.sh
echo "Job running on SLURM NODELIST: $SLURM_NODELIST "
# Modules needed for this SAS job
module purge
module load SAS
#SAS Program execution command
sas /home/user1/SASprog/prog1.sas -sysparm '2006' -log /home/user1//log/proglog2006.log &
sas /home/user1/SASprog/prog1.sas -sysparm '2007' -log /home/user1//log/proglog2007.log &
sas /home/user1/SASprog/prog1.sas -sysparm '2008' -log /home/user1//log/proglog2008.log &
sas /home/user1/SASprog/prog1.sas -sysparm '2009' -log /home/user1//log/proglog2009.log & wait 1) Does accessing the same data in parallel create issues? 2) My understanding is each program will invoke a separate sas session. This should not create conflict in terms of the Work library right? 3) Is there a way to explicitly purge the work libraries at the end of the program? Could there be dumps of earlier work library stashed somewhere I am not seeing, which in turn might be impacting subsequent memory use? After a few trials, I am unable to successfully run the program. Getting the following error: ERROR: Insufficient space in file WORK.REG_DEC.DATA.
ERROR: File WORK.REG_DEC.DATA is damaged. I/O processing did not complete.
WARNING: The data set WORK.REG_DEC may be incomplete. When this step was stopped there were 949663 observations and 178 variables.
WARNING: Data set WORK.REG_DEC was not replaced because this step was stopped. Getting similar error message in all except one log. Should I just reach out to the admin of this cluster regarding memory? Or am I doing something fundamentally wrong. Suggestions are appreciated.
... View more