Filename Pipe X Command -- Limitations?

[jimbarbour]

A few more data points:

1.  I can definitely see intermediate zip archive work files.  See directory excerpt, below.  Notice that both of the intermediate zip archive work files shown froze at the same point, 3,448,766,542 bytes.  

03/03/2020  06:11 PM     3,448,766,542 _Za47800
03/03/2020  06:45 PM     3,448,766,542 _Za75784
03/03/2020  05:41 PM     1,211,498,412 ProvAddresses_Archive_2020-03-03_1714.zip
03/03/2020  05:50 PM     3,283,109,761 ProvAddresses_Archive_2020-03-03_1742.zip
03/03/2020  06:21 PM     2,449,062,651 Rx_Archive_2020-03-03_1759.zip

2.  Here is a second directory excerpt.  I can zip with an X command embedded filename pipe (FILENAME WinOpSys PIPE "&ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""";) files up to  13.9 Gb, but any of the files larger than that freeze.  I said 8 to 10 Gb yesterday.  I was incorrect.  I can go up to about 14 Gb based on additional testing.

02/21/2020  03:12 AM    22,343,712,768 claims_allproviders.sas7bdat
02/20/2020  10:43 PM    19,576,586,240 rx.sas7bdat
02/21/2020  03:12 AM    16,598,983,680 claims_allproviders.sas7bndx
02/20/2020  10:43 PM    13,850,731,520 rx.sas7bndx
02/20/2020  09:22 PM    10,682,236,928 provaddresses.sas7bdat
02/20/2020  07:43 PM     7,105,545,216 providers_all_npitin.sas7bndx
02/20/2020  09:22 PM     6,156,108,800 provaddresses.sas7bndx

@Tom , I like that PIPE embedded within a DATA step.  I haven't used that before.  I suspect a PIPE used in that context will be subject to the same constraint (about 14 Gb apparently) as having a PIPE in a conventional Filename, but I'd love to be wrong.  I will try it and see.  Stay tuned...

@ChrisNZ , not to play favorites, but your code is a bit more different that what I have in place, so I will try your suggestion after Tom's.  Thank you very much for the suggestion; it may well have a good deal of merit inasmuch as it is quite different than my original approach.  And, yes, I could open the archives and see if a) the file is present and b) the file is readable, but this is part of an automated process that will archive a couple hundred files at 0030.  I'm really hoping not to have to do a manual QC here.  🙂

Jim

[ChrisNZ]

> not to play favorites

No worries

> I'm really hoping not to have to do a manual QC 

Mind popular wisdom

A stitch in time...

or is it

Better safe...

?

🙂

[ChrisNZ]

We don't have your code but I assume it's something along the lines of:

data _null_;
  infile WinOpSys;
  input;
  putlog _INFILE_; 
run;

Is that correct?

I don't see why this step should ever time out.

This behavior is very odd.

Any reason you have LRECL in there? Did you have issues about this?

The returned output should be pretty standard.

Lastly, my DOS commands a pretty rusty, but you could maybe try to capture stderr by using :

2>&1

[ChrisNZ]

If worst comes to worst, don't use infile.

Include a redirect-to-file in the command, use call system(), and read the file in  a subsequent step. 

A small hassle for no good reason though.

It looks like a bug to me.

I'd write a small program that creates dummy data and zips it, and then send the whole thing to tech support so they can replicate the problem.

[jimbarbour]

@ChrisNZ ,

Good suggestion.

Indeed, redirecting to a physical file is my workaround for now.  

*			FILENAME	WinOpSys	PIPE		"&ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""";

			%cd(&Zip_Path);
			X	"del WinZip_Messages_Temp.txt & &ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""	>	WinZip_Messages_Temp.txt";
			FILENAME	WinOpSys	"WinZip_Messages_Temp.txt";

I commented out my original FILENAME with the embedded PIPE command and replaced it with:

1.  A call to my change directory macro (%cd) that sets the SAS working directory to the same directory as will hold the Zip Archive.

2.  A stand alone X command with a redirect to a physical text file.  Note that except for the redirect related code, the X command is unchanged from the original PIPE version.

3.  A FILENAME statement with the same name as the original filename.

I then use the same DATA step as before except that I comment out the write of the "Start Zip" to the log since it is completely inaccurate with this set up.

My fear in this context is that I'm really not going to catch errors with my SYMGET('SYSRC'), but hopefully my verifying the presence of "Total bytes=" in the STDOUT is sufficient -- but I'm not really sure.  I have seen wzzip.exe have the occasional error, but I can't reproduce such errors on demand to see if my logic as currently structured will really detect anything.

Jim

[ChrisNZ]

1. Unsure if call system() sets SYSRC. Maybe worth a try(sorry can't test atm)?

2. You could add   2>&1  at the end of the command  to catch STDERR

X "del WinZip_Messages_Temp.txt & &ZipPgmPath. -a ""&Zip_File."" ""&Full_File."" > WinZip_Messages_Temp.txt 2>&1";
			

3. My fear in this context is that I'm really not going to catch errors 

- To really ensure all the zipping worked, you could open the zip file to validate it.

- Note that unless you have a file system that addresses bitrot, your zip file is never perfectly safe.

  btrfs and ZFS are the only two such file systems afaik.

[jimbarbour]

@ChrisNZ , one last thought for today, then I'm off for bed:

The problem seems to be with PIPE. If I run with PIPE, then I'm limited to about 14 Gb.  If I don't use PIPE, then I've yet to find a size limit.  

It's quite possible that i'm missing something, but i'm not seeing where using CALL SYSTEM() would offer an advantage over using the X command.  I suppose for that matter I could use SYSTASK or another equivalent such as the SYSTEM() function or %SYSEXEC.  If i'm missing something here, by all means point it out.  It's late and those few working brain cells that are left to me are going on strike due to the hour.

Jim

[jimbarbour]

@Tom ,

I tried your rather intriguing "filenameless" INFILE with an embedded PIPE.  Alas, it is subject to the same constraints as a PIPE on a FILENAME.  Smaller files work fine.  Larger files work not at all -- they freeze.

Below is a directory excerpt with an additional orphaned intermediate work file, filename _Za46888.  _Za46888 froze at the same point as did the others, 3,488,766,442 bytes.  Note also the number of bytes available on the drive.  Disk space is not an issue here.

03/03/2020  10:09 PM     3,448,766,542 _Za46888
03/03/2020  06:11 PM     3,448,766,542 _Za47800
03/03/2020  06:45 PM     3,448,766,542 _Za75784
03/03/2020  10:05 PM           381,542 Plans_Archive_2020-03-03_2159.zip
03/03/2020  05:41 PM     1,211,498,412 ProvAddresses_Archive_2020-03-03_1714.zip
03/03/2020  05:50 PM     3,283,109,761 ProvAddresses_Archive_2020-03-03_1742.zip
03/03/2020  06:21 PM     2,449,062,651 Rx_Archive_2020-03-03_1759.zip
03/03/2020  10:06 PM     2,449,062,651 Rx_Archive_2020-03-03_2159.zip
               8 File(s) 19,739,414,643 bytes
               2 Dir(s)  3,862,250,860,544 bytes free

Jim

[ChrisNZ]

>Disk space is not an issue here.

Note that the zipping process creates a temporary file that typically does not sit where the final destination is. 

This may not be related to the issue here, but space in the final drive is not sufficient on its own to deem that disk space is not an issue.

See http://kb.winzip.com/help/HELP_DIR.htm

[Tom]

How old is the version of Winzip you are using? Does it suffer from "large file" problem? I thought that limit was 2Gbytes.
Can you try using 7z or some other zip utility?

[jimbarbour]

Tom,

Here is the info on the version of wzzip.exe that I'm using:

+-----------------------------------------------------------------------------------------------------------------+
| WinZip(R) Command Line Support Add-On Version 4.0 64-bit (Build 10562)                                          |
| Copyright (c) 1991-2013 WinZip International LLC - All Rights Reserved                                          |
|                                                                                                                 |
| Updating plans.sas7bdat                                                                                         |
| Total bytes=4653056, Compressed=381416 -> 92 percent savings.                                                   |
|                                                                                                                 |
| Replacing old Zip file "I:\Commercial\monthly_data\Cornerstone\archive\Stg\Plans_Archive_2020-03-03_2232.zip".  |
|                                                                                                                 |
| End Zip at 2020-03-03 23:19:12.01.                                                                              |
| Return code from zipping Plans.sas7bdat = 0.                                                                    |
+-----------------------------------------------------------------------------------------------------------------+

So far, I've been able to zip files that are up to 170 Gb without a problem -- as long as I don't use PIPE in any form.  If I use PIPE, then the limit is somewhere around 14 Gb.  There's a bug with PIPE with wzzip.exe.  The problem appears to lie with SAS, but I can't be completely sure of that.

Jim

[jimbarbour]

@ChrisNZ ,

In answer to your questions:

• The reason I have the %STR("") in the path of wzzip.exe is to prevent the following message.

NOTE: Line generated by the macro variable "ZIPPGMPATH".
29          E:\""Program Files""\WinZip\wzzip.exe
               __
               49
       Setting ZipPgmPath to default value of E:\""Program Files""\WinZip\wzzip.exe
NOTE 49-169: The meaning of an identifier after a quoted string might change in a future SAS release.  Inserting white space 
             between a quoted string and the succeeding identifier is recommended.

• The reason that the double-double quotes are there in the first place is to prevent this error:

  Stderr output:
  'E:\""Program' is not recognized as an internal or external command, operable program or batch file.

  Odd as it may sound, the standard method of quoting, i.e. "&ZipPgmPath" just doesn't work right somehow when passed to the Windows OS.  With the %STR("") placed as it is, everything works, and there are no odd NOTEs put in the log.
• Therefore, this is actually the best coding, however unusual it may be:

  %LET	ZipPgmPath	=	E:\%STR("")Program Files%STR("")\WinZip\wzzip.exe;

• You asked about why I specified an LRECL.  I was trying different things in a "grasping at straws" sort of attempt to fix the limitation with PIPE.  It didn't work.  I removed it.  Please ignore it.
• 2>&1 does in fact re-direct STDERR to STDOUT.  It works, but it didn't unfortunately provide any additional information.  😞  Good suggestion though, and I thank you.
• You are quite correct that temp files in the zipping process don't necessarily reside in the same directory or on the same drive as the final file.  However, I use my %cd macro to set the working directory, and I can see the intermediate work files changing (in that directory) while the zipping is in process.  Beyond that, if I run without a PIPE, there is no problem, even with files 100 X larger.  

Jim

[ChrisNZ]

@jimbarbour I don't have SAS so can't really test this bizarre quote behaviour.

As for the pipe, unless winzip goes crazy, there is little text piped, and the text is not data volume-related, so it's all very odd.

You could pipe directly inside the OS to take SAS out of the equation. For example, this works:
cls | dir "*.log" /s | find " "

It the limitation is with SAS you can then track the incident.

If it is with Windows, you can call MS tech support and they'll create a defect... Just kidding, they don't care. You are a paying subject.

[jimbarbour]

@ChrisNZ , 

Well, I've solved it.  Sort of.  At least I understand what is happening and have developed an improved workaround.

Here's what led me to the solution:

1.  What "causes" the problem?  The use of PIPE.

2.  When does it occur?  When files are over 14 Gb in size.  Insight #1:  It's somehow size related.

3.  What's different when PIPE is used vs. writing to an intermediate text file?  STDERR is automatically re-directed to the SAS log.

4.  Does STDERR increase with the size of the file to be zipped?  Yes, actually, it does.  Wzzip.exe uses STDERR in a non-standard way.  Wzzip.exe shows a series of dots ("...") repeatedly across the console which are used to indicate that work is in progress.  The dots progress from left to right and then jump back to the left without an apparent carriage return.  On the console, the dots appear to take one line.  However, when streamed into a DATA step, they look like the below excerpt.  The larger the file to be zipped, the more there is output that gets written to STDERR.  Somehow, something gets full on SAS's side, and SAS freezes.  I tried playing with MEMSIZE, MEXECSIZE, BUFSIZE, LRECL, etc, but to no avail.

5.  Given that STDERR is redirectable, could STDERR be redirected to an external text file while still piping STDOUT directly in to a DATA step?  Yes, it can, and this is precisely the solution -- or at least improved workaround -- that I am now employing.  

So, the "solution" is to redirect STDERR to an external text file.  This has the following advantages:

1. _RC = SYMGET('SYSRC') should now accurately obtain the return code from wzzip.exe.
2. Start and stop messages now accurately reflect the duration of the run of wzzip.exe.
3. And of course:  Zipping works without a 14 Gb size restriction.

Disadvantages are as follows:

1. STDERR must now be examined manually (or in a subsequent DATA step) if there is a problem and is no longer available to the DATA step controlling the zipping.
2. STDERR exists only in a text file which could be overwritten (and therefore lost), particularly if one were zipping, say, an entire directory, which is precisely my use case.  One could use a text file with a name corresponding to the file being zipped so as to prevent overlay, but, in that case, there would be a proliferation of lots of little STDERR text files.

Jim

SAS code:

%**-----------------------------------------------------------------------------**;
%**								ORIGINAL VERSION								**;
%**	The following file name works but only with files less than about 14 Gb.  	**;
%**	For some reason, larger files "hang" and neither succeed nor fail.  	 	**;
%**	Workaround #1 is to write to an intermediate STDOUT text file.				**;
%**-----------------------------------------------------------------------------**;
FILENAME	WinOpSys	PIPE		"&ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""";

%**-----------------------------------------------------------------------------**;
%**								WORKAROUND #1									**;
%**	The following file name is for Workaround #1.  The original file name is  	**;
%**	split into 1) an independent (no PIPE) "X" command that writes to an   	 	**;
%**	intermediate STDOUT text file and 2) a Filename that reads that STDOUT.		**;
%**-----------------------------------------------------------------------------**;
X	"del WinZip_STDOUT_Temp.txt & &ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""	>	WinZip_STDOUT_Temp.txt";
FILENAME	WinOpSys	"WinZip_STDOUT_Temp.txt";

%**-----------------------------------------------------------------------------**;
%**								WORKAROUND #2									**;
%**	The following file name is for Workaround #2.  STDOUT is piped directly  	**;
%**	into the DATA step but STDERR ("2") is redirected to a STDERR text file.	**;
%**	Apparently, when both STDOUT and STDERR are piped directly into the DATA	**;
%**	step, some buffer gets full, and the entire process freezes.  The downside	**;
%**	of redirecting STDERR to an external text file si that STDERR is then no 	**;
%**	longer available iside the DATA step.  If there is a problem, the external 	**;
%**	text file must be examined manually.										**;
%**-----------------------------------------------------------------------------**;
FILENAME	WinOpSys	PIPE		"&ZipPgmPath. -a ""&Zip_File."" ""&Full_File.""	2>	WinZip_STDERR_Temp.txt";

STDERR:

Searching...            . ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ...............................                               ..............              . . 

[ChrisNZ]

Comprehensive investigative work here Jim, and comprehensive explanations. Hi five! Thank you!

You can mark your post as the solution now. 🙂

And for once this self recognition by the OP will be warranted.

Now to know what causes SAS to freeze when the input buffer fills up.

If you still have stamina to spend on this, send to tech support and they'll create a defect or a Usage Note or both.

To put the fault squarely where it belongs, did you try using an OS-only pipe command?