HI All, I need your suggestion on an issue that our users are experiencing intermittently. From investigation i managed to do until now, they are experiencing I/O errors when transferring SAS tables from work to a file server via the network. This error doesn't appear all the time and that is why it is hard to gather evidence. Platform details: 16 cores server, 128 GB RAM, 250 GB C drive where windows server 2016 is installed, 250 GB D drive where SAS is installed, 3TB S drive where work space is. The throughput is sufficient to and from work drive in terms of SAS requirements and enough free space is on all drives for what the users are doing (using EG clients for manipulating data). Findings until now: - user tries to do a simple data step to transfer a table from work to an external file server folder via network - initial size of the table in work area 93 GB (big 2000 tributes table with 3 million records) the table is uncompressed - destination folder is a COMPRESSED folder on an external file server. - I/O error is returned and destination table is damaged (without possibility of repair using proc datasets) - same error may or may not happen on smaller tables like 40 GB but only if other users are writing in the same folder in the same time - did the same test with the 93 GB table and transferred it to the same file server in an uncompressed folder and it worked fine. - then the user moved manually the table from the uncompressed folder to the compressed one and it decreased size to 24 GB - this is the current workaround. The table is undamaged if transferred in this way. - The network connection was monitored during the transfer and nothing seems to fail, also the process was not interrupted - The user also used x command to transfer the file in the compressed folder and it arrived there damaged as well (which makes me think that the windows process that compresses big files somehow is not working correctly) The question: Naturally I want to say that the conclusion is that when you transfer to a compressed folder using data step that does the compression while you transfer then SAS loses pointer to the rows of the table and ends the file in an unnatural way if the file is too big. What is your opinion? Am I jumping to conclusions? Any other tests I might do?
... View more