Remove Special Characters from Mainframe File which has space in EDIT mode and dots in Browser mode

[rodgerzhang_moh]

Hi there,

Retrieve and output a file in Mainframe using SAS. Various special characters appear in front of some character strings which shift the position of the columns. Looked at mainframe output file, it's empty (space) in EDIT mode but dots can be seen in BROWSER mode. Detail can be seen from attachment. 

Any help is appreciated.

Thanks.

[ballardw]

First thing I would check would be if the program you use the data has an export or report function that creates simple text file either delimited or fixed column output.

It is possible that those characters are being used internally for some formatting purpose and would be best to see if the using program can deal with them instead of creating a work around. If the program doesn't have a special export see if the edit or browse mode have a "print to file" option which may provide a cleaner form to work with.

[jklaverstijn]

I would first investigate the non-printable characters. ISPF Edit displays them as a period. Type the command HEX ON. You will get two lines beneath evey line displaying the first and second half byte in hexadecimal presentation. Look those up in a EBDCDIC table and see if they make sense now. Perhaps they are hex nulls, which makes it easy to deal with. Try to work your way up the delivery chain of the file to see where they came into existence and fix things as high up that chain as possible.

Perhaps some translation did't work out. Real data forensics. I like that.

Hope this helps,

- Jan.

[ChrisNZ]

You can also open the file in a datat step (infile/input) and replace all non-low ascii characters with spaces. For example like this:

data WANT;
  A='AAAA/-,⢶zz';
  put A=;
  A=prxchange('s/[^\x20-\x8D]/ /',-1,A);
  put A=;
run;

A=AAAA/-,⢶zz
A=AAAA/-,   zz

[rodgerzhang_moh]

Thanks everyone.

Viewed with HEX ON, the same - it's a empty (space) in EDIT mode and dot (.) in places where strange characters appear via BROWSER mode when the file is downloaded to PC. It probably has something to do with the format, but can't figure it out by googling.

Also to add is that I can not even read the downloaded text file in PC SAS due to these strange characters saying LOST CARD.

Thanks.

[jklaverstijn]

HEX ON does not affect the content of the actual line. It adds rows underneath that tell you the hexadecimal value of your data. And that will never be empty. Eg a space will be displayed as hex 40. Like this:

Hope this helps to identify your non-printables easier.

Also I am not now trying to fix the issue but to help find out the cause.

Regards, Jan.

[rodgerzhang_moh]

Thanks.

As the dots in front of the fields turn to be strange characters after downloading, I tried and tested moving the position from source file. It seemed that I got the right values after moving 2 positions. So I think we are good now.

Thanks again for everyone's advise and contribution. Much appreciated.

Rodger