Incorrect packed decimal values

[BrianChapman]

I am running a SAS (version 9.4) report on the mainframe (z/OS 2.3) and receiving incorrect packed decimal values in my output.

The report is using MXG to process CICS 110 SMF records. 

Here is the code for the report with the fields as edited numerics (16.6). The fields in question are ELAPSEC and CPUSEC

OPTIONS SOURCE;                                                     
%VMXGSUM(                                                           
   INDATA=PDB.CICSTRAN (KEEP=SMFTIME   APPLID   TRANNAME CPUTM      
                             ELAPSTM   SASDDATE  SMFDT            );
   FORMAT SMFDT     $10.;                                           
   FORMAT SASDATE   DATETIME25.6;                                   
   FORMAT ELAPSEC   16.6;                                           
   FORMAT CPUSEC    16.6;                                           
                                                                    
   ELAPSEC = ELAPSTM;                                               
   CPUSEC  = CPUTM;                                                 
                                                                    
   SASDATE  = SMFTIME;                                              
   SMFDT    = PUT(DATEPART(SASDATE),YYMMDD10.);,                    
                                                                    
   OUTDATA=CICS,                                                    
   SUMBY=SMFDT APPLID TRANNAME,                                     
   SUM=CPUSEC ELAPSEC,    /* SUMS */                                
   FREQ=NUMTRANS,             /* COUNT THE NUMBER OF JOBS */        
   INCODE=                                                          
     IF TRANNAME NOT IN:('AFC');,
   OUTCODE=                                           
     LABEL NUMTRANS='NUMBER*OF*TRANSACTIONS';         
 );                                                  
PROC REPORT DATA=CICS NOWD NOHEADER LS=80;           
OPTIONS NODATE NONUMBER;                             
TITLE1; TITLE2;                                      
COLUMN SMFDT APPLID TRANNAME NUMTRANS CPUSEC ELAPSEC;

Here is the output with edited numerics. This output is correct.

2019-11-20  A70AL001  ADGK        220          0.232033         75.437365
2019-11-20  A70AL001  ADGS       2768          2.782727        810.000074
2019-11-20  A70AL001  DAMY          1          0.001483          0.001914
2019-11-20  A70AL001  IALG       2406         12.758015       8407.511169
2019-11-20  A70AL001  IALK        188          1.040738        618.207749
2019-11-20  A70AL002  ADGS        343          0.299851         29.035216
2019-11-20  A70AL002  DCRY          1          0.000220          0.000558
2019-11-20  A70AL002  IALG        312          1.930452        234.981871

Next, I ran the report with ELAPSEC and CPUSEC as packed decimal (PD8.6).

OPTIONS SOURCE;                                                     
%VMXGSUM(                                                           
   INDATA=PDB.CICSTRAN (KEEP=SMFTIME   APPLID   TRANNAME CPUTM      
                             ELAPSTM   SASDDATE  SMFDT            );
   FORMAT SMFDT     $10.;                                           
   FORMAT SASDATE   DATETIME25.6;                                   
   FORMAT ELAPSEC   PD8.6;                                           
   FORMAT CPUSEC    PD8.6;                                           
                                                                    
   ELAPSEC = ELAPSTM;                                               
   CPUSEC  = CPUTM;                                                 
                                                                    
   SASDATE  = SMFTIME;                                              
   SMFDT    = PUT(DATEPART(SASDATE),YYMMDD10.);,                    
                                                                    
   OUTDATA=CICS,                                                    
   SUMBY=SMFDT APPLID TRANNAME,                                     
   SUM=CPUSEC ELAPSEC,    /* SUMS */                                
   FREQ=NUMTRANS,             /* COUNT THE NUMBER OF JOBS */        
   INCODE=                                                          
     IF TRANNAME NOT IN:('AFC');,
   OUTCODE=                                           
     LABEL NUMTRANS='NUMBER*OF*TRANSACTIONS';         
 );                                                  
PROC REPORT DATA=CICS NOWD NOHEADER LS=80;           
OPTIONS NODATE NONUMBER;                             
TITLE1; TITLE2;                                      
COLUMN SMFDT APPLID TRANNAME NUMTRANS CPUSEC ELAPSEC;

Here is the output. I turned HEX ON to see the associated hex values for each byte. There are non numerics within the packed decimal nibbles.

Also, some of the packed decimal values close (an 8 transposed as a C), but others are wildly inaccurate. The second packed field for A70AL002 DCRY should be 00000000000558C. Instead, it is 0000000000075AC.

2019-11-20  A70AL001  ADGK        220                È  *
FFFF6FF6FF44CFFCDFFF44CCCD44444444FFF44000003034400007335
2019011020001701300100147200000000220000000223C000007476C
---------------------------------------------------------
2019-11-20  A70AL001  ADGS       2768       BÊ@     A   <
FFFF6FF6FF44CFFCDFFF44CCCE4444444FFFF4400002C7744000C0004
2019011020001701300100147200000002768000000722C000001007C
---------------------------------------------------------
2019-11-20  A70AL001  DAMY          1        Ç         J<
FFFF6FF6FF44CFFCDFFF44CCDE4444444444F440000006344000000D4
2019011020001701300100414800000000001000000018C000000011C
---------------------------------------------------------
2019-11-20  A70AL001  IALG       2406       Ì *      Í  Æ
FFFF6FF6FF44CFFCDFFF44CCDC4444444FFFF44000027054400047119
2019011020001701300100913700000002406000001781C000080516E
---------------------------------------------------------
2019-11-20  A70AL001  IALK        188        ËÐ     /B ÈÆ
FFFF6FF6FF44CFFCDFFF44CCDD44444444FFF440000147A440006C079
2019011020001701300100913200000000188000000003C000001274E
---------------------------------------------------------
2019-11-20  A70AL002  ADGS        343       RE       °  %
FFFF6FF6FF44CFFCDFFF44CCCE44444444FFF4400000DC14400009326
2019011020001701300200147200000000343000000295C000002051C
---------------------------------------------------------
2019-11-20  A70AL002  DCRY          1                  ÍÐ
FFFF6FF6FF44CFFCDFFF44CCDE4444444444F4400000020440000007A
2019011020001701300200439800000000001000000002C000000005C
---------------------------------------------------------
2019-11-20  A70AL002  IALG        312        Á       ÑAG 
FFFF6FF6FF44CFFCDFFF44CCDC44444444FFF44000013624400026CC1
2019011020001701300200913700000000312000000905C000003917C

[hashman]

@BrianChapman:

If were trying to track it down, I'd start with comparing the exact actual content of a SAS variable as stored using HEX16. with the value you expect because it is displayed this way by the 16.6 format. For example, if the expected value is 0.000558, I'd print it as:

data _null_ ;                                                                                                                                                                                                                                                   
  v = 0.000558 ;                                                                                                                                                                                                                                                
  put v = hex16. ;                                                                                                                                                                                                                                              
run ; 

On my system it prints as v=3F4248D7E02645E5, but of course on yours (i.e. z/OS) it is different. Make a note of it. Then print ELAPSEC using HEX16. from the record where you have 0.000558 as printed by 16.6. Is the HEX16. value of ELAPSEC the same as printed by the DATA step above?  

Kind regards

Paul D.

[BrianChapman]

Hashman,

I ran the code provided and here is the output:

1           DATA _NULL_ ;     
2             V = 0.000558 ;  
3             PUT V = HEX16. ;
4           RUN ;             
                              
V=3E2491AFC04C8BC9            

After that, I changed my report and made the output fields "HEX16."

Here is that output:

2019-11-20  A70AL001  ADGK        220  403B6689A7000000  424B6FF726000000
2019-11-20  A70AL001  ADGS       2768  412C860CC7000000  4332A0004E000000
2019-11-20  A70AL001  DAMY          1  3E6129D1F6000000  3E7D700000000000
2019-11-20  A70AL001  IALG       2406  41CC20D449000000  4420D782DC000000
2019-11-20  A70AL001  IALK        188  4110A6DCF5000000  4326A352F1000000
2019-11-20  A70AL002  ADGS        343  404CC30D86000000  421D0903ED000000
2019-11-20  A70AL002  DCRY          1  3DE6842741000000  3E24910000000000
2019-11-20  A70AL002  IALG        312  411EE321A5000000  42EAFB5BE8000000

 I see that the first 3 bytes are the same '3E2491'. 

[hashman]

@BrianChapman:

Well, it tells us something, doesn't it? Namely, that the value stored in the SAS variable is not 0.000558 but just 558 (i.e. the time is stored as integer in microseconds). To confirm, run:

DATA _NULL_ ;              
  RETAIN V 0.000558 I 558 ;
  PUT V=HEX16. / I=HEX16. ;
RUN ;                      

How does this step print I=HEX16. on your system? 

[Tom]

At least when running SAS on windows there is difference in the output of PD and S370FPD formats.

159   data test;
160     x= 0.000558;
161     y=put(x,pd8.6);
162     z=put(x,s370Fpd8.6);
163     put x 16.6 / y $hex16. / z $hex16. ;
164   run;

        0.000558
0000000000000558
000000000000558C

Why does there appear to be a call to PROC REPORT in the middle of your code?  It doesn't make much sense to generated packed decimal strings into a report. They are BINARY strings.  Normally to write them to a data file you would just use a data step and a PUT statement.

[BrianChapman]

Tom,

I changed it to output the data using the DATA and PUT statements.

  OPTIONS SOURCE;                                                      
  %VMXGSUM(                                                            
     INDATA=PDB.CICSTRAN (KEEP=SMFTIME   APPLID   TRANNAME CPUTM       
                               ELAPSTM                                 
                               SASDDATE  SMFDT    CPUSEC            ); 
     FORMAT SMFDT     $10.;                                            
     FORMAT SASDATE   DATETIME25.6;                                    
     FORMAT ELAPSEC   16.6;                                            
     FORMAT CPUSEC    16.6;                                            
                                                                       
     ELAPSEC = ELAPSTM;                                                
     CPUSEC  = CPUTM;                                                  
                                                                       
     SASDATE  = SMFTIME;                                               
     SMFDT    = PUT(DATEPART(SASDATE),YYMMDD10.);,                     
                                                                       
     OUTDATA=CICS,                                                     
     SUMBY=SMFDT APPLID TRANNAME,                                      
     SUM=CPUSEC ELAPSEC,    /* SUMS */                                 
     FREQ=NUMTRANS,         /* COUNT THE NUMBER OF TRANSACTIONS */             
     INCODE=                                                           
        IF TRANNAME NOT IN:('AFC');,
     OUTCODE=                                                          
       LABEL NUMTRANS='NUMBER*OF*TRANSACTIONS';                        
    );                                                                 
   DATA _NULL_; SET CICS;                                              
   FILE SASLIST;                                                       
                                                                       
   PUT @001 SMFDT       $10.     /* JOBNAME                          */
       @012 APPLID      $8.      /* VTAM APPLICATION NAME            */
       @022 TRANNAME    $4.      /* TRANSACTION ID                   */
       @028 NUMTRANS     8.      /* NUMBER OF TRANSACTIONS           */
       @038 CPUSEC      PD8.6    /* CPU SECONDS                      */
       @048 ELAPSEC     PD8.6;   /* ELAPSED SECONDS                  */
   RUN;                                                                

I still get invalid packed values.

2019-11-20 A70AL001  ADGK       220                È  *
FFFF6FF6FF4CFFCDFFF44CCCD4444444FFF44000003034400007335
20190110200170130010014720000000220000000223C000007476C
-------------------------------------------------------
2019-11-20 A70AL001  ADGS      2768       BÊ@     A   <
FFFF6FF6FF4CFFCDFFF44CCCE444444FFFF4400002C7744000C0004
20190110200170130010014720000002768000000722C000001007C
-------------------------------------------------------
2019-11-20 A70AL001  DAMY         1        Ç         J<
FFFF6FF6FF4CFFCDFFF44CCDE444444444F440000006344000000D4
20190110200170130010041480000000001000000018C000000011C
-------------------------------------------------------
2019-11-20 A70AL001  IALG      2406       Ì *      Í  Æ
FFFF6FF6FF4CFFCDFFF44CCDC444444FFFF44000027054400047119
20190110200170130010091370000002406000001781C000080516E
-------------------------------------------------------
2019-11-20 A70AL001  IALK       188        ËÐ     /B ÈÆ
FFFF6FF6FF4CFFCDFFF44CCDD4444444FFF440000147A440006C079
20190110200170130010091320000000188000000003C000001274E
-------------------------------------------------------
2019-11-20 A70AL002  ADGS       343       RE       °  %
FFFF6FF6FF4CFFCDFFF44CCCE4444444FFF4400000DC14400009326
20190110200170130020014720000000343000000295C000002051C
-------------------------------------------------------
2019-11-20 A70AL002  DCRY         1                  ÍÐ
FFFF6FF6FF4CFFCDFFF44CCDE444444444F4400000020440000007A
20190110200170130020043980000000001000000002C000000005C
-------------------------------------------------------
2019-11-20 A70AL002  IALG       312        Á       ÑAG 
FFFF6FF6FF4CFFCDFFF44CCDC4444444FFF44000013624400026CC1
20190110200170130020091370000000312000000905C000003917C

[BrianChapman]

I also changed the output format to S370FPD, and I still got the same output.

   PUT @001 SMFDT       $10.        /* JOBNAME                      */
       @012 APPLID      $8.         /* VTAM APPLICATION NAME        */
       @022 TRANNAME    $4.         /* TRANSACTION ID               */
       @028 NUMTRANS     8.         /* NUMBER OF TRANSACTIONS       */
       @038 CPUSEC      S370FPD8.6  /* CPU SECONDS                  */
       @048 ELAPSEC     S370FPD8.6; /* ELAPSED SECONDS              */
   RUN;  

[Tom]

Why do you think the value is wrong?  

Output looks fine to me.

206   /*
207   2019-11-20 A70AL001  ADGK       220                È  *
208   ----+---10----+---20----+---30----+---40----+---50----+----0
209   FFFF6FF6FF4CFFCDFFF44CCCD4444444FFF44000003034400007335
210   20190110200170130010014720000000220000000223C000007476C
211   */
212
213   data test;
214     hexstr='000000077437365C'x;
215     format hexstr $hex16.;
216     x  = input(hexstr,s370fpd8.6);
217     put (_all_) (=);
218   run;

hexstr=000000077437365C x=77.437365
NOTE: The data set WORK.TEST has 1 observations and 2 variables.

Certainly looks like what the SAS doucmentaion says that PD format generates when running on Z/OS.

https://documentation.sas.com/?docsetId=hosto390&docsetTarget=n1shzbsiufgrzin1f3q38pdsfpw7.htm&docse...

[BrianChapman]

The value is wrong. 77437365C is a valid packed value, but it's the wrong value for the data. Take a look at my original post. I posted the accurate edited numeric values. It's supposed to be 75437365C.

Also, take a look at the second packed decimal column for the last row (A70AL002 IALG). It's supposed to be 234981871C. Instead it is 2369C1C71C. The third digit is wrong and the '8' digits have been transposed as 'C'. Packed decimal values can only have alphabetic values as the low order digit (typically ending in 'C' or 'D').

[Tom]

---

@BrianChapman wrote:

The value is wrong. 77437365C is a valid packed value, but it's the wrong value for the data. Take a look at my original post. I posted the accurate edited numeric values. It's supposed to be 75437365C.

Also, take a look at the second packed decimal column for the last row (A70AL002 IALG). It's supposed to be 234981871C. Instead it is 2369C1C71C. The third digit is wrong and the '8' digits have been transposed as 'C'. Packed decimal values can only have alphabetic values as the low order digit (typically ending in 'C' or 'D').

---

You will need to debug this yourself. 

To start just subset the data to one o the example values that looks wrong and make sure exactly what value the variable has for that observation.

data _null_;
  SET CICS;
  where SMFDT ='2019-11-20'
    and APPLID= 'A70AL002'
    and TRANNAME ='IALG'
  ;
  test1=put(ELAPSEC,PD8.6);
  test2=put(ELAPSEC,PK8.6);
  put ELAPSEC / ELAPSEC HEX16. / ELAPSEC 17.6 / ELAPSEC BEST17.6 / test1 $hex16. / test2 $hex16.;
run;

It the values in the log for TEST1 look like what you want then check and make sure there is not some other process that is modifying the file before you look at it.  For example if you are moving that file using something like an FTP process and it is treating it as TEXT instead of BINARY then it will get mangled.

Looking at the values you are talking about here is what I see:

250   data test;
251     do hexstr='000000075437365C'x,'000000077437365C'x,'000000234981871C'x;
252       num = input(hexstr,s370fpd8.6);
253       put hexstr=$hex16. num=17.6 num=best17.6 ;
254     end;
255   run;

hexstr=000000075437365C num=75.437365 num=75.437365
hexstr=000000077437365C num=77.437365 num=77.437365
hexstr=000000234981871C num=234.981871 num=234.981871

[hashman]

@Tom:

Yes, there is - because you're not running under z/OS.

On non-z/OS platforms, PDw.d renders the same as PKw.d, i.e. neither places the sign marker in the last nibble, while s370FPDw.d does (C or D,depending on plus or minus), and so does s370FPDUw.d (unsigned, places F in the last nibble).

On z/OS, PDw.d and s370FPDw.d are exactly the same (they write the sign) but different from PKw.d (it does not). 

[Tom]

So he either needs to use PK format to write or leave the values as is and use S370FPD informat to read.

[BrianChapman]

Yes. PD and S370FPD behave the same on z/OS. SAS is generating the numbers in proper form, but the values are wrong.

77437365C is a valid packed value, but it's the wrong value for the data. It's supposed to be 75437365C.

[Tom]

Note that '8187'x in EBCDIC are the letters 'ag'. While 'C1C7'x in EBCDIC are the letters 'AG'.

Did something try to UPCASE() the characters in your file?

[BrianChapman]

Tom,

The input data has not been modified.