topic Re: Can pipe option and encoding= option take effects at the same time? in SAS Programming

Can pipe option and encoding= option take effects at the same time?

whymath — Thu, 11 Nov 2021 05:55:58 GMT

My operation system is encoding as EUC-CN and I work on SAS DMS in Unicode enviroment. I'm trying to write a program to get the directory of some path on my compute. My first version of code like this:

%let path = %sysfunc(pathname(sasuser));
x dir "&path." /b/s > "&path.\_dir_.txt" & exit;

data test;
  infile "&path.\_dir_.txt" truncover encoding = 'zeuc';
  input path$1024.;
run;

x del "&path.\_dir_.txt" & exit;

It works. But I think there is a more effective and beatiful way, so I rewrite it. My second version of code like this:

data test;
  infile "dir ""&path."" /b/s" pipe truncover encoding = 'zeuc';
  input path$1024.;
run;

It's much shoter now, but is grammarly wrong:

ERROR 23-2: option name "encoding" is invalid.

How do you think of that? Can pipe option and infile= option take effects at the same time?

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Thu, 11 Nov 2021 07:26:14 GMT

It seems that device pipe and option encoding= cannot be used at the same time, though that is not specified in the documentation.

You might want to ask tech support to have this clarified.

There is also a Feedback link all pages, including the relevant one: https://documentation.sas.com/doc/en/vdmmlcdc/1.0/lestmtsref/n1rill4udj0tfun1fvce3j401plo.htm#n0tskmdejsgmk0n1ex2dc5411mjk

They are usually very good at replying.

Does the infile statement not get the correct data if the encoding is not specified?

Re: Can pipe option and encoding= option take effects at the same time?

whymath — Thu, 11 Nov 2021 08:29:19 GMT

Thanks, @ChrisNZ.

The link doesn't describe the pipe and encoding inssue, perhaps I have to ask my tech support.

pipe option actually pass the command, which is dir "&path." /b/s, to the terminal of operation system(CMD for Windows). However, CMD's results are encoded as ANSI, for me, that means EUC-CN. So I'm reading a EUC-CN encoded file in unicode enviroment and I can't specify the encoding option. There are a lot of garbled characters in the result.

Re: Can pipe option and encoding= option take effects at the same time?

Kurt_Bremser — Thu, 11 Nov 2021 08:44:30 GMT

The issue may be documented in a quite veiled way:

(from the INFILE Statement)

ENCODING= 'encoding-value'

specifies the encoding to use when reading from the external file.

(emphasis by me)

This could indicate that ENCODING= can only be used with files, and not other device types.

Although I see no technical reason for this, as the combination of PIPE and ENCODING= works in a FILENAME statement:

filename test pipe "ls" encoding="WLATIN1";

data _null_;
infile test;
input;
put _infile_;
run;

filename test clear;

Which means that @whymath can simplify the code by moving the PIPE option to the FILENAME statement, thereby avoiding the creation of the external file.

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Thu, 11 Nov 2021 09:39:57 GMT

Interesting @Kurt_Bremser . I did try that before replying and this fails on Win64 with 9.4M7.

So it might be that Unix behaves differently. All the more reason to ask Tech Support to sort this out, and to have the documentation amended.

Re: Can pipe option and encoding= option take effects at the same time?

whymath — Thu, 11 Nov 2021 09:42:22 GMT

I still get the same error when I run your first line of program.

This is my info of lab:
Version: SAS9.04.01M5

Machine: SAS-Win10-0029 X64_10PRO

Anything wrong?

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Thu, 11 Nov 2021 10:02:39 GMT

> CMD's results are encoded as ANSI,

@whymath That's disappointing.

Have you try loading the correct codepage? UTF-8 is 65001 for example.

So you could try running something like

chcp 65001 & dir "&path" /b/s

chcp 65001>nul & dir "&path" /b/s

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Thu, 11 Nov 2021 09:58:31 GMT

@whymath It seems this only works on Unix, not on Windows.

Again, a good reason to seek clarification from tech support, and to request clarification of the doc.

Re: Can pipe option and encoding= option take effects at the same time?

Kurt_Bremser — Thu, 11 Nov 2021 11:09:34 GMT

It is definitely an OS issue, because this also works on AIX:

data _null_;
infile "ls" pipe encoding="WLATIN1";
input;
put _infile_;
run;

Just did not think of testing it earlier.

@whymath you should clearly check this with SAS technical support. If it is an issue with the way Windows handles pipes, then it should at least be documented.

In the Windows companion, there is only an error message documented when ENCODING= is used with CARDS or DATALINES.

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Thu, 11 Nov 2021 22:26:57 GMT

This looks like a defect, it's very uncommon that such syntax is inconsistently supported across platforms.

Re: Can pipe option and encoding= option take effects at the same time?

s_lassen — Fri, 12 Nov 2021 09:20:05 GMT

The documentation for the FILENAME statement explicitly states:

Not all device types support the encoding option. For more information, see the documentation for your operating system.

But there is no mention of a limitation on using ENCODING= with the PIPE device type in the Windows documentation. There probably should be.

Re: Can pipe option and encoding= option take effects at the same time?

ChrisNZ — Sat, 13 Nov 2021 02:27:41 GMT

Indeed. And such support depending o the OS is rather unusual. So this might not have happened on purpose.

Re: Can pipe option and encoding= option take effects at the same time?

Tom — Sat, 13 Nov 2021 04:32:00 GMT

Did you try doing the transcoding yourself?

data test;
  infile "&path.\_dir_.txt" truncover ;
  input path $1024.;
  path=kcvt(path,'zeuc','utf-8');
run;