Call System to run a python program with a redirect to a text file

[jmspy]

Hello-

I have a bash shell script that works perfectly fine when executing it from the UNIX command line, however when trying to execute it from SAS it ignore the redirects.

Here's the shell script:

#!/bin/bash

while IFS="" read -r idnumber|| [ -n "$idnumber" ]
do
curl -k -s -H 'placeholderheader' | \
/env/python -c "import sys, json; print('"$idnumber",' + str(json.load(sys.stdin)['Clients'][0]['ClientId']))" \
>> idfile.txt
done < sourcefile.txt

Executing this in terminal produces an output to that text file without issue, but that does not happen when executing from SAS:

	data _null_;
		call system('/mydirectory/python/ClientID/clientcurl.sh');
	run;

Runs without SAS error, but nothing is appended to the text file.  I understand that the "instance" that SAS creates to execute this system command is different to how one uses terminal, but does anyone know if there is a way to get around this issue?

Thanks!

[Kurt_Bremser]

You did not specify the absolute path names to idfile.txt and sourcefile.txt.

Your SAS process has a different current working directory, so you need the absolute path.

[jmspy]

I apologize, I do have the absolute paths in there I just trimmed it for security's sake 🙂

[Kurt_Bremser]

Run the external command with a pipe:

filename oscmd pipe '/mydirectory/python/ClientID/clientcurl.sh 2>&1';

data _null_;
infile oscmd;
input;
put _infile_;
run;

and see if the SAS log reveals anything.

[jmspy]

I attempted the method you posted, and I receive this error in the output:

raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

My assumption is that the way I am redirecting the source file could be the culprit, especially since it works perfectly in terminal:

done < /mydirectory/sourcefile.txt

The instance that runs this script seems to not be able to reference that source file, but I don't know enough about how SAS executes system tasks like this.

[Tom]

See if your SAS session can read the file.  Try reading the first 10 lines.

data _null_;
  infile '/mydirectory/sourcefile.txt' obs=10;
  input;
  list;
run;

[smantha]

Does sas user have permissions to run system calls? Then there is the absolute path thing that @Kurt_Bremser pointed out.

[jmspy]

Yes I can execute scripts that run smb client and native python scripts (where I don't redirect outputs)