About agoldma

agoldma · ‎06-28-2014

I don't know the definition of "&tvar_sasmindat"dt, so I used the constant "01jan1582 00:00:00"dt (it looks like this is your own way of handling date-time constants prior to 1582) Main idea: if you want to use an [existing-informat] in your value-range-set(s), you have to use the (DEFAULT= ) option. This is documented in SAS Help: FORMAT Procedure > INVALUE Statement (near the bottom) Only one (DEFAULT= ) option is allowed per informat-name (per invalue statement), so if you want multiple lengths, you need either 2 separate informats or use the Function Complier: PROC FCMP (similar to a macro). Example #12 in SAS Help for Proc Format "Creating a Function to Use as a Format" is a very good example for combining a user-defined format with a user-defined function. proc fcmp outlib=work.functions.smd; function in19(x $) ; return(input(substr(x,1,19),ymddttm19.)); endsub; function in26(x $) ; return(input(x,ymddttm26.)); endsub; run; proc format lib=work; invalue sas_datetime_frmt '0000-00-00-00.00.00.000000' -< '1582-01-01-00.00.00.000000' = '01jan1582 00.00.00'dt '1582-01-01-00.00.00.000000' -< '1926-01-01-00.00.00.000000' = [in19()] '1926-01-01-00.00.00.000000' -< '2095-01-01-00.00.00.000000' = [in26()] '2095-01-01-00.00.00.000000' - '9999-12-31-23.59.59.999999' = [in19()] other = .; run; The next 2 data steps (work.format_in and work.format_out) are the same as yours. One possible idea for date-time values prior to 1/1/1582 is a special missing value such as A. or _.

agoldma · ‎06-28-2014

PDw.d informat can read shorter-length strings than 5 bytes if it's used with a (default= ) option in the proc format. See reminders by jwillis and Jap in the Proc Format - user-defined informat PDw.d in Windows discussion. This is documented in SAS Help: FORMAT Procedure > INVALUE Statement Solution for this discussion in z/OS is: proc format; invalue inhext (default=2) other=[pd2.] ; run;

agoldma · ‎06-28-2014

Jaap, thank you for suggesting the use PROC FCMP It seems to be similar to a macro, but apparently you can use it inside an informat. It can do some pre-processing while reading data from a text file... very interesting, and.... that example #12 in Proc Format "Creating a Function to Use as a Format" shows the use the (DEFAULT=) option with parentheses. Jwillis, if you like macros, you might like PROC FCMP better. Looks like it allows you to do everything that a macro does but with easier syntax.

agoldma · ‎06-28-2014

Jwillis, I'm responding your post # 16 The use of brackets [] in PROC FORMAT is documented in SAS Help: FORMAT Procedure > INVALUE Statement as "[existing-informat]" ... right next to the place that says to use parentheses around the "DEFAULT= option" with an existing-informat, so I knew I had to use it, but I missed the parentheses (max= option is not necessary; I used it because the default= option didn't work) You're absolutely right that without the parentheses, SAS interprets "default=3 max=3" as 2 more value-range-sets. That's why it wasn't giving me an error.

agoldma · ‎06-28-2014

Jwillis, Your example (in post # 15) shows one example where an informat could be useful -- giving the user of the program a choice of which informat to use. My situation is different. I have inconsistent data in a text file (with some compressed fields). In the same location on different records I could have things like: '800012'x, '000000'x, '202020'x, ' ', '123', 'TOM' ... all kinds of stuff. Without using my own user-defined informat, I could still read this data. I would pause my INPUT line with @; use an IF statement to check the contents of the input buffer, and then read it appropriately. However, I think that using my own informat is at least more elegant if not faster -- to do the same kind of conditional reading.

agoldma · ‎06-25-2014

Data_null_, I'm sorry for unmarking your previous responses as correct and helpful, but the real problem was in the parentheses. Jwillis had it right. I knew I had to use the default= option because SAS Help said so, but I didn't use it correctly, and unfortunately SAS didn't give me an error. I'm also interested in proc fcmp. Thank you, jaap, for suggesting it. I'll try to make it work later.

agoldma · ‎06-25-2014

Jwillis, I'll try to look at your code later today (that's a lot of code, glad to see that you're interested in dinasaurs) If you start with my original code and change 2 things that data_null_ suggested, the user-defined-informat-method should work The 2 things are: use "other" instead of "low-high" in the definition of the informat use "truncover" on the input line Also... I think you're right about using parentheses for the options (deault=3 max=3) I'll try to look at it later today

agoldma · ‎06-24-2014

Jaap, I agree that every byte is split into 2 nibbles, and the sign is in the first byte (unfortunately in Windows it takes a full byte to store the sign). Still, this leaves plenty of room to fit the value of -12 into the "pd3." format. Even pd2. has enough room to store the value of -12. If you run through the code in my original post, you'll see that it includes a data step where the pd3. informat works perfectly as long as I don't use PROC FORMAT to define my own informat, so the problem is not in the use of the pd3. informat. It's in using it inside the proc format to create my own informat. If I write something with a pd3. format, the right way to read it is with a pd3. informat. As some previous posts implied, the PROC FORMAT doesn't handle some lengths correctly for user-defined informats, so we have to take additional measures. If you still think that I assumed something incorrectly, please specify which assumption it is. Also, your suggestion about using the fcmp function sounds interesting, but I couldn't find that function. Can you please check the spelling of "fcmp"

agoldma · ‎06-24-2014

Data _null_, thank you very much for both suggestions: using "other" instead of low-high and using truncover. Both were necessary. I don't see why they should be necessary, but I'm glad it works. This official Help article uses low-high as an example: Specifying Values or Ranges

agoldma · ‎06-24-2014

Jwillis, thank you for putting some serious effort into this. The example that I provided seems pointless, but it's sufficient to illustrate the behavior of SAS that appears to be inconsistent with documentation. This example is not the real code that I'm using in production, but it gives the same error. This is not just a theoretical exercise because user-defined formats allow us to do fast and streamlined testing of the data prior to reading it. This is very useful for reading inconsistent data. There are many papers written on proc format (easy to find on the internet). Here are some examples in these forums: Must be a better way to do this (missing values) Converting Raw Files with Dynamically Calculated Layout Proc format issues: invalue statement ignores length specified with informats Coincidentally, the third link above seems to be describing a similar problem that I'm having. I didn't find this link prior to posting my question because I was looking for the PDw.d informat. I didn't know that this problem applied to other informats. Since SAS is very versatile, there are many ways to accomplish the same thing. My current workaround is to break the input statement in half and do the testing in the middle there. It works, but I was hoping for a nicer solution.

agoldma · ‎06-23-2014

Jwillis, thank you for trying to run through this. The text file is generated by the first data step. (I'm not using any data that's not in my original post) The "-12 € " is the way it looks on one line -- the "€ " is the packed representation of the value -12 The first data step generates one record with two representations of the value -12 (the regular representation and the packed representation separated by a space). Please try running the second data step, which reads that text file correctly Please try running the third data step, which generates an error, and... look at the log In the log, you'll see the HEX representation of the packed value -12

agoldma · ‎06-23-2014

The sample data is generated in the first data step. There's only one record, so I don't see why there's an issue with the carriage return. I don't have any difficulty using the pd3. informat -- as illustrated in the second data step, which runs perfectly well without any special carriage returns. I get this error only when using PROC FORMAT to define my own informat.

agoldma · ‎06-23-2014

SAS version 9.3 I can write and read data with a pd3. informat as long as I don't use my own user defined informat. I would like to use my own informat because I would like to do some testing prior to reading. I define my own informats like this: proc format; invalue inthree default=3 max=3 low-high = [3.] ; invalue inpdthree default=3 max=3 low-high = [pd3.] ; run; This data step writes 3 packed bytes in positions 5:7 data _null_; x=-12; file 'C:\PROD\SAS93\test3.txt'; put @1 x 3. @5 x pd3. ; run; This data step reads 3 packed bytes from positions 5:7 data test3a; infile 'C:\PROD\SAS93\test3.txt'; input @1 x1 3. @5 x2 pd3. ; run; proc print data=test3a; run; This data step produces a lost card error: data test3b; infile 'C:\PROD\SAS93\test3.txt'; input @1 x1 3. @5 x2 inpdthree. ; run; Does anyone know why I'm getting the lost card error? This data step reads x1 correctly but produces an invalid data error for x2 in positions 5-7: data test3b; infile 'C:\PROD\SAS93\test3.txt'; informat x1 inthree. x2 inpdthree. ; input @1 x1 @5 x2 ; run; Does anyone know why I'm getting the invalid data error for x2?

agoldma · ‎06-21-2014

The Definition of Missing Values recognizes "missing character value as a blank space", so I would rather say that SAS considers the space character " " to be equivalent to an empty character string "". This is the part that I forgot. Thank you for reminding me about this. If "" = " ", and " "='20'x in ASCII and '40'x in EBCDIC, then there are many opportunities for some packed bytes to be smaller than '20'x and even more opportunities to be smaller than '40'x. Also, I should've read the last 2 sentences of Order of Missing Values in SAS Help. In my specific example, the only thing that I'm trying to avoid is spaces, so I should've excluded only spaces instead of a range of characters: if substr(_infile_,1,3) ne '' then input @1 var1 pd3. ;

agoldma · ‎06-20-2014

Most text books say that the smallest values are the missing values: ""(character) and .(numeric) Apparently, this applies only to regular unpacked data. The byte that represents packed data seems to be smaller than the usual missing ""(character) value. Such bytes don't have any meaning and can appear to be symbols like: Null, start of heading, start of text, end of text, punch off, horizontal tab... I thought that such symbols would be greater than missing "", but apparently they're less than or equal to missing. This is in z/OS EBCDIC using SAS 9.2, but I expect similar results in Windows ASCII. Specific example: trying to read a packed decimal field conditionally: if substr(_infile_,1,3)>'' then input @1 var1 pd3. ; This line doesn't produce an error, but the IF expression is always false, so it looks like packed bytes are less than or equal to missing. Please correct me if I'm wrong.

Online Status	Offline
Date Last Visited	‎06-09-2020 11:53 AM

Re: stop EG upon first error

Re: Unexpected Size of Dataset

Re: Unexpected Size of Dataset

Re: Unexpected Size of Dataset

Re: Unexpected Size of Dataset

Re: Unexpected Size of Dataset

Unexpected Size of Dataset

Re: Difference between NODUPKEY and NODUPRECS?

Re: Hot fix Y39009

Re: Picture Format - fractional seconds

Define prompt default value using Macro Variabe

Re: Making EG Stop Executing subsequent process flows based on a value...

Re: Unexpected Size of Dataset

Re: Unexpected Size of Dataset

Re: How To Work Efficiently with Very Big SAS Datasets

Re: Dealing with 32 character restrictions

Re: EG: Stop locking files!

Re: Difference between NODUPKEY and NODUPRECS?

SAS EG - Clear Results

Re: Moving Average with Multiple Observations

Re: Proc format issues: invalue statement ignores length specified wit...

Re: proc format - PDw.d informat in z/OS - range of bytes in a user de...

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Re: Proc Format - user-defined informat PDw.d in Windows

Proc Format - user-defined informat PDw.d in Windows

Re: Smallest values with packed data

Smallest values with packed data