Job started using client ID authcode starts then fails

[Malux]

I am trying to execute a sas via 3.5 job using a client id and secret.  I can see the job start successfully using the authenticated user.

However, while the job starts, it fails shortly after with the following error:

The SAS program provided for the job completed execution with at least one error.
500 : ""errorCode":30029,"message":"Failed to launch process.","details":["traceId: 916dbe177de581e0","path: /launcher/processes","path: /compute/contexts/3466208e-7633-48cd-9073-c2d47394c868/sessions"],"errors":["errorCode":30013,"message":"Failed to launch process: host=<HOST> port=44743 ssl=true","links":[],"version":2,"httpStatusCode":0,"errorCode":30012,"message":"Server error: status=0x80BFD100 messages=[ERROR: Access denied.]","links":[],"version":2,"httpStatusCode":0],"links":[],"version":2,"httpStatusCode":500"

Note: I hid the server name for security.

If, however, I restart the job from the environment manager by right-clicking and clicking "Execute" the job completes successfully.

Failure:

Success after retry:

Quick note: I need to hide the user name for security.  

 

Below is how I am creating the client_id:

 curl -k -X POST "<SERVER>/SASLogon/oauth/clients" \
  -H "Authorization: Bearer $BEARER" \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "<USER>",
    "client_secret": "<SECRET>",
    "authorities": ["openid","uaa.user", "uaa.resource","sasapp", "sastoken","jobExecution", "session", "compute", "jobExecution.launch","job","files","casManagement","data","identity","read","write","execute","sas.jobs","sas.compute","sas.files","sas.identities","sas.data","sas.read","compute","audit","admin","authorization","credentials","jobDefinition"],
    "scopes": ["openid","uaa.user", "uaa.resource","sasapp", "sastoken","jobExecution", "session", "compute", "jobExecution.launch","job","files","casManagement","data","identity","read","write","execute","sas.jobs","sas.compute","sas.files","sas.identities","sas.data","sas.read","compute","audit","admin","authorization","credentials","jobDefinition"],
    "authorized_grant_types": ["authorization_code","client_credentials","refresh_token"],
    "redirect_uri": ["http://localhost:5000/oauth/callback"],
    "autoapprove": true
  }'

Note: user, server and secret have been obfuscated.

here's how the login is happening after getting the auth code:

curl -k <SERVER>/SASLogon/oauth/token \
     -H "Accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" \
     -u "<USER>:<SECRET>" \
     -d "grant_type=authorization_code&code=<CODE>"

here's how the job is launched:

curl -X POST "<SERVER>/jobExecution/jobs" \
  -H "Authorization: Bearer $BEARER" \
  -H "Content-Type: application/json" \
  -d '{
        "jobDefinitionUri": "/jobDefinitions/definitions/<URI>"
      }'

I've also tried specifying the compute context.  all to no avail.

Keep in mind: I do get a response showing the job started, but it fails immediately after launch.

[arthurdpereira]

Hi @Malux , how are you?

From your description, it seems that the job execution is being triggered before the SAS authentication is fully established. As a result, the first attempt fails, but when you retry (now with an active authenticated session), it works.

I’d suggest adding a short wait (around 10 seconds) after obtaining the token and before executing the job. This should give enough time for the authentication to be fully recognized by SAS before the job starts.

Hope this helps!

[Malux]

@arthurdpereira I'm not sure it's that.  it looks like I can't establish a session

curl -k -X POST \

  "<SERVER>/compute/contexts/<CONTEXT/sessions" \ 

  -H "Authorization: Bearer $BEARER" \

  -H "Content-Type: application/json" \

  -d '{}'

the above returns:

ErrorResponse[version=2,status=500,err=30029,id=<null>,message=Failed to launch process.,detail=[traceId: f7d207e6db5f2611, path: /launcher/processes, path: /compute/contexts/<CONTEXT>/sessions],remediation=<null>,targetUri=<null>,errors=[ErrorResponse[version=2,status=0,err=30013,id=<null>,message=Failed to launch process: host=<HOST> ssl=true,detail=<null>,remediation=<null>,targetUri=<null>,errors=<null>,links=[]], ErrorResponse[version=2,status=0,err=30012,id=<null>,message=Server error: status=0x80BFD100 messages=[ERROR: Access denied.],detail=<null>,remediation=<null>,targetUri=<null>,errors=<null>,links=[]]],links=[]]

[gwootton]

You appear to be successfully authenticating to Viya using an authorization code, so your REST API request to run the job is successful. This triggers a request from job-execution to compute to run the code. To do so, compute attempts to start a compute server through a request to the launcher service, which would submit the attempt to the launcher server.
The "Access Denied" error is coming from the launcher server when authenticating against the host.
If you store a valid user ID and password to launch the process on the compute server host for your identity under Environment Manager > My Credentials in the DefaultAuth domain, is the launch successful?

[Malux]

@gwootton YES!  the launch is successful if I enter my user credentials there.

curiously the job execution works:

curl -X POST "<SERVER>/jobExecution/jobs" \

  -H "Authorization: Bearer $BEARER" \

  -H "Content-Type: application/json" \

  -d '{

        "jobDefinitionUri": "/jobDefinitions/definitions/<URI>"

      }'

BUT...creating a session gives me a 401 now not a 500 now

curl -X POST \                                                                

  "<SERVER>/compute/contexts/<CONTEXT>/sessions" \

  -H "Authorization: Bearer $BEARER" \ 

  -H "Content-Type: application/json" \

  -d '{}'                        

ErrorResponse[version=2,status=401,err=401,id=HTTP_401,message=HTTP/1.1 401 Unauthorized,detail=[path: /compute/contexts/<CONTEXT>/sessions, traceId: 4296d285b1cc8cba],remediation=,targetUri=<null>,errors=[],links=[]]%    

also: I can't seem to get anything from _webout files...

[gwootton]

A 401 unauthorized response usually suggests an issue with the token you're using (whereas 403 forbidden means you have a permissions issue). You may need to get a fresh token.

For the _webout issue you might want to check _omitSessionResults is set to false.

Control When the Compute Provider Saves Results
https://go.documentation.sas.com/doc/en/sasadmincdc/v_064/caljobs/p1xrrmzo4afphbn1pn3j9hqfd5rv.htm#n...