topic Re: Call System to run a python program with a redirect to a text file in SAS Programming

Call System to run a python program with a redirect to a text file

jmspy — Wed, 24 Jun 2020 18:59:29 GMT

Hello-

I have a bash shell script that works perfectly fine when executing it from the UNIX command line, however when trying to execute it from SAS it ignore the redirects.

Here's the shell script:

#!/bin/bash

while IFS="" read -r idnumber|| [ -n "$idnumber" ]
do
curl -k -s -H 'placeholderheader' | \
/env/python -c "import sys, json; print('"$idnumber",' + str(json.load(sys.stdin)['Clients'][0]['ClientId']))" \
>> idfile.txt
done < sourcefile.txt

Executing this in terminal produces an output to that text file without issue, but that does not happen when executing from SAS:

	data _null_;
		call system('/mydirectory/python/ClientID/clientcurl.sh');
	run;

Runs without SAS error, but nothing is appended to the text file. I understand that the "instance" that SAS creates to execute this system command is different to how one uses terminal, but does anyone know if there is a way to get around this issue?

Thanks!

Re: Call System to run a python program with a redirect to a text file

Kurt_Bremser — Wed, 24 Jun 2020 19:08:36 GMT

You did not specify the absolute path names to idfile.txt and sourcefile.txt.

Your SAS process has a different current working directory, so you need the absolute path.

Re: Call System to run a python program with a redirect to a text file

jmspy — Wed, 24 Jun 2020 19:23:09 GMT

I apologize, I do have the absolute paths in there I just trimmed it for security's sake 🙂

Re: Call System to run a python program with a redirect to a text file

Kurt_Bremser — Wed, 24 Jun 2020 19:32:32 GMT

Run the external command with a pipe:

filename oscmd pipe '/mydirectory/python/ClientID/clientcurl.sh 2>&1';

data _null_;
infile oscmd;
input;
put _infile_;
run;

and see if the SAS log reveals anything.

Re: Call System to run a python program with a redirect to a text file

smantha — Wed, 24 Jun 2020 19:35:50 GMT

Does sas user have permissions to run system calls? Then there is the absolute path thing that @Kurt_Bremser pointed out.

Re: Call System to run a python program with a redirect to a text file

jmspy — Wed, 24 Jun 2020 21:20:18 GMT

Yes I can execute scripts that run smb client and native python scripts (where I don't redirect outputs)

Re: Call System to run a python program with a redirect to a text file

jmspy — Wed, 24 Jun 2020 21:45:46 GMT

I attempted the method you posted, and I receive this error in the output:

raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

My assumption is that the way I am redirecting the source file could be the culprit, especially since it works perfectly in terminal:

done < /mydirectory/sourcefile.txt

The instance that runs this script seems to not be able to reference that source file, but I don't know enough about how SAS executes system tasks like this.

Re: Call System to run a python program with a redirect to a text file

Tom — Thu, 25 Jun 2020 03:20:53 GMT

See if your SAS session can read the file. Try reading the first 10 lines.

data _null_;
  infile '/mydirectory/sourcefile.txt' obs=10;
  input;
  list;
run;