topic Re: TCP Attempt Fails in Administration and Deployment

TCP Attempt Fails

PabloJ — Thu, 22 Mar 2018 15:51:10 GMT

Hi all,

I have a recurrent alert showed on Environment Manager, the alert is:

Alert Properties

Name:	TCP Attempt Fails	Priority:	! - Low
Resource:	xxxxx.xxxxxx	Alert Date:	22-mar-2018 16:20:00
Description:	Attempts to Establish Connections to a TCP service. Should be close to zero.	Alert Status:	NOT FIXED

Condition Set

If Condition:	Tcp Attempt Fails per Minute > 20,0% of 20,4 (Baseline Value)
Actual Value:	20,0

Enable Action(s):	Each time conditions are met.

I don´t know how to manage this alert and what is the issue but the alert is being generated every 10 minutes.

Thanks in advance,

Kind regards.

Re: SAS Backup failed

JuanS_OCS — Thu, 22 Mar 2018 15:38:39 GMT

Hello @PabloJ,

what resource is this alert set for? This would help a lot.

With this information, I can only say that the alert is for a network attempt to see if a service is alive. As long as there are several attempts and failed, it believes one service is down. Hence, the resource that is being monitored, either is down or blocking the access to that network port number.

Since the topic is about the SAS Backup, I guess the monitored resource is the SAS Deployment Agent? SAS Deployment Backup tool requires all SAS Deployment Agents to be UP and reachable each other.

Without knowing more, this is all I can tell for now, sorry.

Re: TCP Attempt Fails

SASKiwi — Thu, 22 Mar 2018 19:26:25 GMT

Is this on a SAS VA installation or something else? What version and maintenance release of SAS does this apply to? We have the same problem on our SAS VA servers and have a track open with SAS on this.

Re: TCP Attempt Fails

PabloJ — Fri, 23 Mar 2018 09:01:13 GMT

thanks both for your replies. We have SAS 9.4 installed and VA 7.4 version on a Linux server. I don´t know what resource is affected by this alert I would like to know some way to detect that and then fix it but environment manager don´t give enough information.

The alert is showed every 10 minutes.

Re: TCP Attempt Fails

JuanS_OCS — Fri, 23 Mar 2018 09:31:47 GMT

Hello @PabloJ, @SASKiwi,

SAS Environment Manager is based on VMWare Hyperic product. As such, you will be able to find and answer a lot of questions giving a look into the product documentation pages. And the agents are based on the hyperic/sigar technology, hence those product pages can help to better understand the messages (sigar will basically do the checks with netstat, fyi)

For this purpose:

https://pubs.vmware.com/vfabric5/index.jsp#com.vmware.vfabric.hyperic.4.6/Local_Platform_Services.html which points to

https://pubs.vmware.com/vfabric5/index.jsp?topic=/com.vmware.vfabric.hyperic.4.6/TCP_Socket_Platform_Service.html

Besides this, interestingly, I think it is worth a quick view to something just unirectly related, the VMWare Gemfire product pages: https://pubs.vmware.com/vfabric5/index.jsp#com.vmware.vfabric.gemfire.6.6/managing/monitor_tune/slow_receivers_preventing_problems.html which talks about slow TCP/IP receivers.

Anyway, I believe it is worth a good network analysis of your servers, to check if there are TCP packages that are getting lost or simply blocked. It could be. OS commands such as tcdump or netstat or more advanced network tools as Wireshark can be of great help.

Last bits, some documentation from SAS, just informative in this case:

https://support.sas.com/rnd/emi/SASEnvMgr/EVSAF/All_Alert_Definitions.pdf

http://support.sas.com/documentation/cdl/en/evug/69029/HTML/default/viewer.htm#n0urxjgvbkxyo5n1dyp1onad56nq.htm

Re: TCP Attempt Fails

SASKiwi — Fri, 23 Mar 2018 23:14:16 GMT

@PabloJ @JuanS_OCS:

So far SAS Tech Support have pointed me at this SAS note which I have tried: http://support.sas.com/kb/58/589.html

They have also asked me to tweak JVM heap sizes. Nothing so far has been a reliable fix. I'm also getting Garbage Collection alerts as well, although they are not as frequent as the TCP alerts.

Re: TCP Attempt Fails

JuanS_OCS — Sun, 25 Mar 2018 14:59:40 GMT

Hi there @SASKiwi,

interesting. I feel curious. Did any of those proposed changes had any positive or negative impact on the alerts that you receive?

Re: TCP Attempt Fails

SASKiwi — Tue, 27 Mar 2018 19:08:37 GMT

Increasing heap sizes tended to increase Garbage Collection alerts but reduce TCP alerts but they still occurred frequently. So far nothing I have tried has really made a big difference.

Re: TCP Attempt Fails

JuanS_OCS — Wed, 28 Mar 2018 07:20:20 GMT

Thank you @SASKiwi, @PabloJ,

That is interesting. Heapsizes and GC is intended only for JVMs, although this alert in the thread is intended at a host level, not service/JVM level.

A question: have you checked the size of the audit tables in the WIP database? Perhaps a good idea to clean/archive this data.

Re: TCP Attempt Fails

PabloJ — Wed, 28 Mar 2018 09:17:00 GMT

Hi all,

I tried cleaning the audit tables but the problem persist.
Also tried with tcpdump and whireshark but both give a lot of information that its difficult to find the cause of the issue.

I opened a ticket on SAS Support but they only suggested me to ask here (in SAS Community) or/and talk with my company network systems department.

I will keep trying to find the solution of my problem. Thanks a lot.

Re: TCP Attempt Fails

JuanS_OCS — Wed, 28 Mar 2018 14:18:08 GMT

Hello @PabloJ,

I think your network engineers can help a lot. If they cannot, perhaps you could hire an especialised SAS Consultant, for 1 day perhaps. Never underestimate the synergies between a SAS Consultant and your IT teams 🙂

The reason is because I think this kind of question it can only be answered on-site and probably by more than one discipline working together.

Re: TCP Attempt Fails

SASKiwi — Wed, 28 Mar 2018 18:56:40 GMT

Archiving the audit tables had no impact on this issue.