SAS VIYA 3.5 Install Issues

[vorkady]

We are encountering install issue  with rabbitmq cluster.   following are the partial  error logs . Any help appreciated

    "stdout": "Fri Feb 12 14:55:37 EST 2021 setup_rabbit_cluster *************

              Fri Feb 12 14:55:37 EST 2021 setup_rabbit_cluster Beginning SAS RabbitMQ clustering setup

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster SSL is trueFri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster CHECK_PORT is 5671

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster First host in list is server6.srv.bmogc.net

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster First host shortname is server6

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster Primary Host is server6.srv.bmogc.net\

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster Primary Short Host is server6

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster My Host is saseocbldapp007.srv.bmogc.net\

              Fri Feb 12 14:55:38 EST 2021 (debug) setup_rabbit_cluster I am a clustering host\

              Fri Feb 12 14:55:38 EST 2021 setup_rabbit_cluster Copy generated Erlang shared secret to Rabbit.

              Fri Feb 12 14:55:42 EST 2021 setup_rabbit_cluster Copying generated Erlang shared secret to Rabbit.

              Fri Feb 12 14:55:42 EST 2021 setup_rabbit_cluster Starting sas-viya-rabbitmq-server-default

              Fri Feb 12 14:55:59 EST 2021 setup_rabbit_cluster Wait for a Rabbit listener instance on the primary host server6.srv.bmogc.net

              Fri Feb 12 14:55:59 EST 2021 (debug) setup_rabbit_cluster wait_for_host server6.srv.bmogc.net succeeded, listener detected.

              Fri Feb 12 14:56:02 EST 2021 (debug) setup_rabbit_cluster Issuing clustering commands

              Fri Feb 12 14:56:02 EST 2021 (debug) setup_rabbit_cluster Stop the Rabbit app (not the server though)

              Fri Feb 12 14:56:05 EST 2021 (debug) setup_rabbit_cluster USE_LONGNAME = true, MY_PRIMARY is rabbit@server6.srv.bmogc.net

              Fri Feb 12 14:56:05 EST 2021 (debug) setup_rabbit_cluster Cluster with the primary node

              Fri Feb 12 14:56:08 EST 2021 setup_rabbit_cluster Attempts to join_cluster with the node rabbit@server6.srv.bmogc.net failed. Clustering failed.

              Fri Feb 12 14:56:08 EST 2021 setup_rabbit_cluster Returned status was 69",

[SColeman]

Hi Vorkady, 

The logs in /var/log/sas/viya/rabbitmq-server/default/* and /tmp/sas_setup_rabbit_cluster.log should contain more information. 

Without knowing what those logs contain, we can't be certain what the issue is, but there are two common causes behind rabbit clustering errors:

1. DNS problems, such as when multiple NICs exist on the machines. This can be resolved by using host_vars as seen here:

https://communities.sas.com/t5/SAS-Communities-Library/Understanding-Network-Binding-Variables-in-SA... 

You can create a file where the filename is the ansible alias of that machine, IE: viya_node1.yml and the contents of the file contain your network config as seen here: 

$ cat /path/to/sas_viya_playbook/host_vars/sasviya01.yml
---
network_conf:
  SAS_HOSTNAME: sasviya01.race.sas.com
  SAS_BIND_ADDR_IF: "eth0"

I would also make sure that the /etc/hosts files between the machines have consistent values. 

2. Port blockages. IE: firewalld doesn't have the necessary items in its whitelist or iptables rules are filtering the necessary ports. This can be resolved by ensuring your ports aren't blocked.