BookmarkSubscribeRSS Feed
RyanSimmons
Pyrite | Level 9

I have a directory of files that I am trying to read into SAS. I am using a macro that has always worked in the past without error on this same directory. Now, inexplicably, it has stopped working properly. The relevant part of the code is, simply, this:

 

libname infiles 'H:\DGHI\DHS\Data\';

DATA _indiv;
	set infiles.&dind;
	keep caseid bidx v024 v002 v005 v006 v007 v008 v009 v010 v011 v012 v025 v113 v116 v136 v137 v149 v190 v201 v501 bord b0 b1 b2 b3 b4 b5 b6 b7 b8 b11 b12 b13;
run;

Where "&dind" is the macro variable for the name of the input dataset. There are 14 pre-existing SAS datasets in the directory, all of which need to be read into SAS. 13 of them are working normally, and 1 of them has stopped working (inexplicably ... two weeks ago I ran this exact code on these files with no problems). When I run the above code on this file (which is called KEBR70FL), it gives me:

 

ERROR: File INFILES.KEBR70FL.DATA does not exist. 

 

Further, if I run PROC DATASETS on INFILES, the file KEBR70FL does not show up! It only shows 13 of the 14 SAS datasets. However, if I navigate to the directory specified by INFILES in Windows Explorer, I can see that the KEBR70FL dataset DOES in fact, exist in that directory. 

 

Now, here is where things get even stranger. The 13 other datasets in that directory (the 13 that are recognized by SAS as existing) I can all right-click and open in either SAS 9.4 OR Sas Enterprise Guide 7.11, just as normal. However, the 14th dataset doesn't even have an option for opening it in SAS 9.4 when I right-click the file! It ONLY lets me open the dataset in Enterprise Guide (and it opens there succesfully, without error). 

 

So, I right-clicked this file and maneuevered to "Open with..." and selected SAS 9.4 in the window that pops up when you use that option. However, it doesn't open the file at all. Nothing happens in SAS 9.4, but when I look back in Windows Explorer after having done this, it has now changed the file type from a SAS dataset to an SD2 file, and output a text document full of error messages (I've copied those at the below). 

 

Even STRANGER is the fact that doing this changed a COMPLETELY DIFFERENT DATASET to an SD2 file as well! One of the 13 files that had previously been working is now suddenly changed to an SD2 file, simply by trying to open this other file in SAS 9.4 (despite it working in Enterprise Guide, as I mentioned)! Not only that, but it changed THE BACKUP COPIES OF THESE DATASETS THAT WERE LOCATED IN A DIFFERENT DIRECTORY! After the file got changed to SD2, I went to the backup directory to retrieve the original SAS dataset version and somehow it had ALSO been changed to an SD2, despite not being connected to any of the above operations in anyway that I can tell.

 

EDIT: Even worse, the backup file that was overwritten into an SD2 was in a compressed (zip) folder in a different directory, and it was still somehow altered by just trying to open this one file in SAS. It also altered a completely different backup file in a different compressed folder in a different directory. 

 

Does anybody have the slightest idea what is going on, here? I don't understand at all how this dataset got corrupted so that SAS 9.4 wouldn't recognize it anymore (yet it was still recognized by Enterprise Guide), and I CERTAINLY don't understand how any of the above was able to independently corrupt other files located in completely different directories...

 

(Here are the error messages in the text file that is created along with the SD2 file:

 

NOTE: Unable to open SASUSER.REGSTRY. WORK.REGSTRY will be opened instead.
NOTE: All registry changes will be lost at the end of the session.

WARNING: Unable to copy SASUSER registry to WORK registry. Because of this, you will not see registry customizations during this 
         session.
NOTE: Unable to open SASUSER.PROFILE. WORK.PROFILE will be opened instead.
NOTE: All profile changes will be lost at the end of the session.
NOTE: Copyright (c) 2002-2012 by SAS Institute Inc., Cary, NC, USA. 
NOTE: SAS (r) Proprietary Software 9.4 (TS1M3) 
      Licensed to DUKE UNIVERSITY - T&R - SFA - NCICU GRANT, Site 70082794.
NOTE: This session is executing on the X64_7PRO  platform.



NOTE: Updated analytical products:
      
      SAS/STAT 14.1
      SAS/ETS 14.1
      SAS/OR 14.1
      SAS/IML 14.1
      SAS/QC 14.1

NOTE: Additional host information:

 X64_7PRO WIN 6.1.7601 Service Pack 1 Workstation

NOTE: SAS initialization used:
      real time           0.21 seconds
      cpu time            0.18 seconds
      
WARNING: A line of input contains one or more null (hexadecimal 00) characters, which might cause 
         the SAS System to ignore part of the line.  The line, with null characters printed as 
         question marks, is: 
SAS     6.08.04 WIN      3.10                                   STATTRAN
1          SAS     6.08.04 WIN      3.10                                   STATTRAN  ÀL;
           ___
           180

ERROR 180-322: Statement is not valid or it is used out of proper order.

1                                                                                        ÑA•c‘T2Æ
                                                                                         __
                                                                                         180
1        ! #8Û 8                      ¬Q    ÀL;

ERROR 180-322: Statement is not valid or it is used out of proper order.

1          8Û 8                      ¬Q    ÀL;

_                                                                                                 

180

2                                         The SAS System              11:59 Friday, April 29, 2016

ERROR 180-322: Statement is not valid or it is used out of proper order.

1          ÑAwm%w­©\·!êh                                                                         

____                                                                                              

180
1        !                                                    s       `7          è6  x       x6  
1        ! p       6  p       ˜5  p       (5  p       ¸4  p       H4  p       Ø3  p       h3  p  
1        !      ø2  p       ˆ2  p       2  p       ¨1  p       81  p       È0  p       X0  p     
1        !   è/  p       x/  p       /  p       ˜.  p       (.  p       ¸-  p       H-  p       Ø
1        ! ,  p       h,  p       ø+  p       ˆ+  p       +  p       ¨*  p       8*  p       È)  
1        ! p       X)  p       è(  p       x(  p       (  p       ˜'  p       ('  p       ¸&  p  
1        !      H&  p       Ø%  p       h%  p       ø$  p       ˆ$  p       $  p       ¨#  p     
1        !   8#  p       È"  p       X"  p       è!  p       x!  p       !  p       ˜   p       (
1        !    p       ¸  p       H  p       Ø  p       h  p       ø  p       ˆ  p         
1        ! p       ¨  p       8  p       È  p       X  p       è

ERROR 180-322: Statement is not valid or it is used out of proper order.

ERROR: Errors printed on pages 1,2.

NOTE: SAS Institute Inc., SAS Campus Drive, Cary, NC USA 27513-2414
NOTE: The SAS System used:
      real time           0.31 seconds
      cpu time            0.18 seconds
      

)

11 REPLIES 11
Kurt_Bremser
Super User

Do you have your Windows Explorer configured to show the file extensions?

I have a suspicion that somehow those datasets were written with the SAS 6 engine.

RyanSimmons
Pyrite | Level 9

Yes, I do. And no, all of these datasets were written with SAS 9.4. As I said, I ran this exact code on these exact datasets only two weeks ago and didn't encounter any of these problems.

 

Interestingly, the ORIGINAL versions of these datasets (which were downloaded from dhsprogram.org) were SAS 6 files, and I had to convert them into SAS 9 files so I could use them. But I did that conversion MONTHS ago, and have used the SAS 9 versions of the datasets multiple times (literally hundreds, actually) over that span without any issues or errors or corruption until today.

 

EDIT: And what the $%*&$*(%&*$ I just noticed that whatever is going on here HAS EFFECTED SAS DATASETS LOCATED ON A DIFFERENT SERVER! How is this even possible?! 

Kurt_Bremser
Super User

Writing a dataset from 9.4 does not automatically mean that the V9 engine is used. If SAS thinks that a library is a V6 library, it will use that engine when writing datasets.

So I guess that SAS was somehow tricked into thinking V6, or a non-converted dataset made it into your library (repeated download, restore from backup? )

 

RyanSimmons
Pyrite | Level 9

I am aware of the issues with V9 vs V6 libraries. It was a major issue for me when I was first converting these files. However, as I said, I solved that issue MONTHS ago, and haven't had any issues with it in the ensuing time period. That directory has not been modified in any way since 3/11/2016 according to my logs. 

RyanSimmons
Pyrite | Level 9

Whatever happened with those SAS datasets I will probably never know. I remained unable to open them (even with explicitly using the v6 option in the LIBNAME statement). The only way I was able to get around this was by redownloading the raw (version 6) SAS datasets form the original source and reconvert them into v9 datasets. But this still offers me no clues as to how this whole corruption thing happened in the first place, or why it seemed to have a cascading effect on unrelated files in different directories. 

FreelanceReinh
Jade | Level 19

Hi Ryan,

 

What you describe as a "cascading effect" is probably harmless. I think, this is the usual effect that occurs when the association of a file type (file name extension) with an application is changed. When you used the "Open with..." dialog, you probably did not uncheck the checkbox "Always use the selected program to open this kind of file" -- which caused the association to change.

 

But this did not modify all these files (with the same file name extension). They are just displayed differently in Windows Explorer (the description in column "Type" and the file icon change) and, of course, they are opened differently when double-clicked. (I think the only "physical" change occurs in the Windows registry.) These changes, indeed, affect all files of the same type including zipped files and files on different drives, servers, etc. But as soon as you change the file type association to what it was before, everything should be back to normal.

 

In the case of SAS datasets, the association with SAS 9.4 does not seem to make sense: It appears that SAS then tries to "execute" the datasets as if they were SAS programs (see the error messages you posted).

RyanSimmons
Pyrite | Level 9

The cascading effect was not harmless, however. It wasn't just the display that was different, the FILE EXTENSION WAS CHANGED. They were changed from .sas7bdat files to .sd2 files! And, as I said in my last post, the only way I could fix this was by deleting them and redownloading the raw data! It went far beyond just changing the icon called by Windows Explorer, it completely eliminated my ability to even ACCESS those files, period (even declaring V6 option in the LIBNAME statement). And it wouldn't let me change the file type association back; in fact, right-clicking these .sd2 files didn't even give me the "Open with" option anymore! 

 

 

As for your last sentence:

 

"In the case of SAS datasets, the association with SAS 9.4 does not seem to make sense: It appears that SAS then tries to "execute" the datasets as if they were SAS programs (see the error messages you posted)."

 

What do you mean by "the association with SAS 9.4 does not seem to make sense"? What part of the error message indicates it is trying to run them as a program? (I am just trying to understand the error log; I'm not an expert on this part of SAS)

Reeza
Super User

Honestly, what you're saying doesn't seem possible. 

 

SAS can't just change things like that. Was anything externally changed on your computer, an IT upgrade or something else?

Perhaps they did a backup and restore that wasn't communicated due to an issue?

RyanSimmons
Pyrite | Level 9

I agree with you that it doesn't seem possible. That's why I posted here, because I legitimately cannot understand how or what happened with those files. 

 

I don't know what external changes could have been made to either my computer or the servers in question. None that have been clearly communicated to me, at least. 

 

It looks like I am just going to have to file a ticket with SAS Support and talk to our IT department.

FreelanceReinh
Jade | Level 19

@RyanSimmons wrote:

(...) 

What do you mean by "the association with SAS 9.4 does not seem to make sense"? What part of the error message indicates it is trying to run them as a program? (I am just trying to understand the error log; I'm not an expert on this part of SAS)


When I select SAS 9.4 in the "Open with..." dialog of a SAS dataset (.sas7bdat) -- after unchecking the checkbox "Always use ..."! -- I get a log file similar to that you posted. It is named <name_of_the_dataset>.sas7bdat.log, located in the same folder as the .sas7bdat file and it is the SAS log resulting from an attempt to execute the dataset as a program.

 

All those cryptic characters are the binary content of the .sas7bdat file, senselessly interpreted as ASCII characters and mixed with a few human-readable words and numbers such as the SAS version (SAS 6.08 in your case, SAS 9.04 in mine). You can see the same "gibberish" content if you open a (small!) SAS dataset with a text editor (such as Notepad). Not surprisingly, the SAS compiler's typical reaction to this assumed "program code" is:

ERROR 180-322: Statement is not valid or it is used out of proper order.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 11 replies
  • 2075 views
  • 4 likes
  • 4 in conversation