Strange error message calling proc http

[LaurieF · Accepted solution]

After much discussion with SAS Global Hosting, they came up with a solution. We had originally discounted the -nofiles value, because both the service account and my userid had the default value of 350,000. But someone had the bright idea of checking what LSF was doing - and found that it was overriding the value either at (LSF) startup or when a job was being submitted. That value was 4,500. By removing that restriction and restarting LSF, the jobs now run to completion.

Except for an API nextPage issue which has cropped up, which will keep me busy today.

If it were easy, somebody would've already fixed it...

Ngā mihi nui,

Laurie

[LaurieF]

Yet a better fix. Changing the Linux setting fixed the problem, but adding no_conn_cache as a parameter to the proc http call stops the connections being retained. There's a small overhead apparently (When you enable NO_CONN_CACHE, you forgo all benefits of cached connections, including cached authentication. Each call that uses authentication must re-authenticate, which can take time) but the effect is empirically minimal. I'm still getting four gets a second as before.

[Kurt_Bremser]

So the procedure seems to keep an open connection, but still creates a new one each time it is called. I see that as a bug.

[LaurieF]

I disagree - that's not what the description of the parameter implies at all. As I said, it caches the connections, as many processes do, making the it more efficiently. Since I am calling proc http sometimes hundreds of thousands of times a day, including in a single job, I want it to run as efficiently as possible.

In effect, turning it off hardly made a dent in my run-times, and according to SAS Global Hosting, the open connection counts from my jobs have dropped right off. I can imagine however situations where keeping the connections and credentials cached could be an advantage.

[Kurt_Bremser]

But why should it cache connections when it obviously doesn't reuse them? That doesn't make sense.