TCP Attempt Fails

[PabloJ]

Hi all,

I have a recurrent alert showed on Environment Manager, the alert is:

Alert Properties

Name:	TCP Attempt Fails	Priority:	! - Low
Resource:	xxxxx.xxxxxx	Alert Date:	22-mar-2018 16:20:00
Description:	Attempts to Establish Connections to a TCP service. Should be close to zero.	Alert Status:	NOT FIXED

Condition Set

If Condition:	Tcp Attempt Fails per Minute > 20,0% of 20,4 (Baseline Value)
Actual Value:	20,0
Enable Action(s):	Each time conditions are met.

I don´t know how to manage this alert and what is the issue but the alert is being generated every 10 minutes.

Thanks in advance,

Kind regards.

[JuanS_OCS]

Hello @PabloJ,

what resource is this alert set for? This would help a lot.

With this information, I can only say that the alert is for a network attempt to see if a service is alive. As long as there are several attempts and failed, it believes one service is down. Hence, the resource that is being monitored, either is down or blocking the access to that network port number.

Since the topic is about the SAS Backup, I guess the monitored resource is the SAS Deployment Agent? SAS Deployment Backup tool requires all SAS Deployment Agents to be UP and reachable each other.

Without knowing more, this is all I can tell for now, sorry.

[SASKiwi]

Is this on a SAS VA installation or something else? What version and maintenance release of SAS does this apply to? We have the same problem on our SAS VA servers and have a track open with SAS on this. 

[PabloJ]

thanks both for your replies. We have SAS 9.4 installed and VA 7.4 version on a Linux server. I don´t know what resource is affected by this alert I would like to know some way to detect that and then fix it but environment manager don´t give enough information.

The alert is showed every 10 minutes.

[JuanS_OCS]

Hello @PabloJ, @SASKiwi,

SAS Environment Manager is based on VMWare Hyperic product. As such, you will be able to find and answer a lot of questions giving a look into the product documentation pages. And the agents are based on the hyperic/sigar technology, hence those product pages can help to better understand the messages (sigar will basically do the checks with netstat, fyi)

For this purpose:

https://pubs.vmware.com/vfabric5/index.jsp#com.vmware.vfabric.hyperic.4.6/Local_Platform_Services.ht... which points to

https://pubs.vmware.com/vfabric5/index.jsp?topic=/com.vmware.vfabric.hyperic.4.6/TCP_Socket_Platform...

Besides this, interestingly, I think it is worth a quick view to something just unirectly related, the VMWare Gemfire product pages: https://pubs.vmware.com/vfabric5/index.jsp#com.vmware.vfabric.gemfire.6.6/managing/monitor_tune/slow... which talks about slow TCP/IP receivers.

Anyway, I believe it is worth a good network analysis of your servers, to check if there are TCP packages that are getting lost or simply blocked. It could be. OS commands such as tcdump or netstat or more advanced network tools as Wireshark can be of great help.

Last bits, some documentation from SAS, just informative in this case:

https://support.sas.com/rnd/emi/SASEnvMgr/EVSAF/All_Alert_Definitions.pdf

http://support.sas.com/documentation/cdl/en/evug/69029/HTML/default/viewer.htm#n0urxjgvbkxyo5n1dyp1o...

[SASKiwi]

@PabloJ@JuanS_OCS:

So far SAS Tech Support have pointed me at this SAS note which I have tried: http://support.sas.com/kb/58/589.html

They have also asked me to tweak JVM heap sizes. Nothing so far has been a reliable fix. I'm also getting Garbage Collection alerts as well, although they are not as frequent as the TCP alerts.

[JuanS_OCS]

Hi there @SASKiwi,

interesting. I feel curious. Did any of those proposed changes had any positive or negative impact on the alerts that you receive?

[SASKiwi]

Increasing heap sizes tended to increase Garbage Collection alerts but reduce TCP alerts but they still occurred frequently. So far nothing I have tried has really made a big difference.

[JuanS_OCS]

Thank you @SASKiwi, @PabloJ,

That is interesting. Heapsizes and GC is intended only for JVMs, although this alert in the thread is intended at a host level, not service/JVM level.

A question: have you checked the size of the audit tables in the WIP database? Perhaps a good idea to clean/archive this data.

[PabloJ]

Hi all,

I tried cleaning the audit tables but the problem persist.
Also tried with tcpdump and whireshark but both give a lot of information that its difficult to find the cause of the issue.

I opened a ticket on SAS Support but they only suggested me to ask here (in SAS Community) or/and talk with my company network systems department.

I will keep trying to find the solution of my problem. Thanks a lot.

[JuanS_OCS · Accepted solution]

Hello @PabloJ,

I think your network engineers can help a lot. If they cannot, perhaps you could hire an especialised SAS Consultant, for 1 day perhaps. Never underestimate the synergies between a SAS Consultant and your IT teams 🙂

The reason is because I think this kind of question it can only be answered on-site and probably by more than one discipline working together.

[SASKiwi]

Archiving the audit tables had no impact on this issue.