Infile / File Considerably Slower in 9.3

jklein_271 — Fri, 02 Sep 2011 15:37:08 GMT

@SAS Experts:

I've recently migrated a project that bad been running with EG 4.3 / SAS 9.2 to EG 4.3 / SAS 9.3. Overall, I've been very pleased with SAS 9.3 so far. However, I've noticed drastically increased run times for data _null_ processes involving infile / file statements. The first process uses a data _null_ with recfm=s and a file statement to pull a zip file from the internet byte for byte (well blocks of bytes) and the run time increased from less than 10 minutes to 30 minutes consistently. The second process is converting a csv with some PERL functions to a cleaned txt file using the infile / file in a data _null_ as well. The run time with this process increased from an average of 1 hour 30 mins to 2 hours 15 mins. Considering nothing else has changed in our environent, I'm leaning towards the move to 9.3. In addition, as I am seeing the same or less run times in every other process not involving infile / file, I'm specifically thinking a default system option changed impacting the buffer, blksize, etc. I've messed with the blksize to see the same run times, but the file just waiting to write until it hits 1M instead of the default 8K. Does anyone have any thoughts or possible changes made in 9.3 that could impact these processes? At a minimum, I was hoping to compare the default options in 9.2 to 9.3 side by side. As I don't have 9.2 installed or licensed anymore, would anyone be able to give me a Friday gift and output the 9.2 proc options results and attach to the thread? A user in an environment with no custom options set in the sasv9.cfg or autoexec would obviously be the best case scenario, but I'll take anything.

Thanks for your time.

Infile / File Considerably Slower in 9.3

Tom — Fri, 02 Sep 2011 20:20:56 GMT

Can you check if the default settings for buffers have changed?

Infile / File Considerably Slower in 9.3

jklein_271 — Sat, 03 Sep 2011 00:19:58 GMT

Are there any buffer system options that would apply to the infile / file processing? I'm seeing a few that relate to datasets, view, indexes, etc. Does anyone have any specific system options I should be focusing on for flat file processing?

Infile / File Considerably Slower in 9.3

boschy — Mon, 17 Oct 2011 23:34:47 GMT

What happens when you try to read these files on the operating system, e.g. using shell/command scripts without SAS involved?

Also, in your SAS data _null_ step, I'd try reading the files into a dataset without any processing. See how long that takes.

topic Infile / File Considerably Slower in 9.3 in SAS Enterprise Guide

Infile / File Considerably Slower in 9.3

Infile / File Considerably Slower in 9.3

Infile / File Considerably Slower in 9.3

Infile / File Considerably Slower in 9.3