topic SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" in Administration and Deployment

topic SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" in Administration and Deployment https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558630#M16254 <P>When trying to deploy the SAS Viya 3.4 I'm receiving </P><P> </P><DIV>2019-05-08 19:17:53,744 p=1481 u=root |  TASK [consul-x64_redhat_linux_6-yum : Execute confirm consul can be accessed] **********************************************************************************************</DIV><DIV>2019-05-08 19:17:53,745 p=1481 u=root |  task path: /sas/install/sas_viya_playbook/roles/consul-x64_redhat_linux_6-yum/tasks/start.yml:76</DIV><DIV>2019-05-08 19:17:53,799 p=1481 u=root |  Read vars_file '../vars.yml'</DIV><DIV>2019-05-08 19:17:53,836 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:53,855 p=1481 u=root |  Read vars_file '../vars.yml'</DIV><DIV>2019-05-08 19:17:53,893 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:53,951 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:54,176 p=1481 u=root |  fatal: [Service_Layer_I]: FAILED! => {</DIV><DIV>    "changed": true,</DIV><DIV>    "cmd": [</DIV><DIV>        "bash",</DIV><DIV>        "-c",</DIV><DIV>        "source /opt/sas/viya/config/consul.conf && /opt/sas/viya/home/bin/sas-bootstrap-config status leader"</DIV><DIV>    ],</DIV><DIV>    "delta": "0:00:00.086451",</DIV><DIV>    "end": "2019-05-08 19:17:54.137918",</DIV><DIV>    "invocation": {</DIV><DIV>        "module_args": {</DIV><DIV>            "_raw_params": "bash -c 'source /opt/sas/viya/config/consul.conf && /opt/sas/viya/home/bin/sas-bootstrap-config status leader'",</DIV><DIV>            "_uses_shell": false,</DIV><DIV>            "argv": null,</DIV><DIV>            "chdir": null,</DIV><DIV>            "creates": null,</DIV><DIV>            "executable": null,</DIV><DIV>            "removes": null,</DIV><DIV>            "stdin": null,</DIV><DIV>            "warn": true</DIV><DIV>        }</DIV><DIV>    },</DIV><DIV>    "msg": "non-zero return code",</DIV><DIV>    "rc": 1,</DIV><DIV>    "start": "2019-05-08 19:17:54.051467",</DIV><DIV>    "stderr": "Get https://localhost:8501/v1/status/leader: dial tcp 127.0.0.1:8501: getsockopt: connection refused\nERROR: Unable to determine the address of the Raft leader for the data center in which the Consul agent is running.",</DIV><DIV>    "stderr_lines": [</DIV><DIV>        "Get https://localhost:8501/v1/status/leader: dial tcp 127.0.0.1:8501: getsockopt: connection refused",</DIV><DIV>        "ERROR: Unable to determine the address of the Raft leader for the data center in which the Consul agent is running."</DIV><DIV>    ],</DIV><DIV>    "stdout": "",</DIV><DIV>    "stdout_lines": []</DIV><DIV> </DIV><DIV>Consul logs specifies "==> bind_addr cannot be empty" I have added "<SPAN>consul_bind_adapter = eth0 </SPAN>" in inventory.ini file.</DIV> Tue, 14 May 2019 14:07:25 GMT stalasani 2019-05-14T14:07:25Z SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558630#M16254 <P>When trying to deploy the SAS Viya 3.4 I'm receiving </P><P> </P><DIV>2019-05-08 19:17:53,744 p=1481 u=root |  TASK [consul-x64_redhat_linux_6-yum : Execute confirm consul can be accessed] **********************************************************************************************</DIV><DIV>2019-05-08 19:17:53,745 p=1481 u=root |  task path: /sas/install/sas_viya_playbook/roles/consul-x64_redhat_linux_6-yum/tasks/start.yml:76</DIV><DIV>2019-05-08 19:17:53,799 p=1481 u=root |  Read vars_file '../vars.yml'</DIV><DIV>2019-05-08 19:17:53,836 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:53,855 p=1481 u=root |  Read vars_file '../vars.yml'</DIV><DIV>2019-05-08 19:17:53,893 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:53,951 p=1481 u=root |  Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py</DIV><DIV>2019-05-08 19:17:54,176 p=1481 u=root |  fatal: [Service_Layer_I]: FAILED! => {</DIV><DIV>    "changed": true,</DIV><DIV>    "cmd": [</DIV><DIV>        "bash",</DIV><DIV>        "-c",</DIV><DIV>        "source /opt/sas/viya/config/consul.conf && /opt/sas/viya/home/bin/sas-bootstrap-config status leader"</DIV><DIV>    ],</DIV><DIV>    "delta": "0:00:00.086451",</DIV><DIV>    "end": "2019-05-08 19:17:54.137918",</DIV><DIV>    "invocation": {</DIV><DIV>        "module_args": {</DIV><DIV>            "_raw_params": "bash -c 'source /opt/sas/viya/config/consul.conf && /opt/sas/viya/home/bin/sas-bootstrap-config status leader'",</DIV><DIV>            "_uses_shell": false,</DIV><DIV>            "argv": null,</DIV><DIV>            "chdir": null,</DIV><DIV>            "creates": null,</DIV><DIV>            "executable": null,</DIV><DIV>            "removes": null,</DIV><DIV>            "stdin": null,</DIV><DIV>            "warn": true</DIV><DIV>        }</DIV><DIV>    },</DIV><DIV>    "msg": "non-zero return code",</DIV><DIV>    "rc": 1,</DIV><DIV>    "start": "2019-05-08 19:17:54.051467",</DIV><DIV>    "stderr": "Get https://localhost:8501/v1/status/leader: dial tcp 127.0.0.1:8501: getsockopt: connection refused\nERROR: Unable to determine the address of the Raft leader for the data center in which the Consul agent is running.",</DIV><DIV>    "stderr_lines": [</DIV><DIV>        "Get https://localhost:8501/v1/status/leader: dial tcp 127.0.0.1:8501: getsockopt: connection refused",</DIV><DIV>        "ERROR: Unable to determine the address of the Raft leader for the data center in which the Consul agent is running."</DIV><DIV>    ],</DIV><DIV>    "stdout": "",</DIV><DIV>    "stdout_lines": []</DIV><DIV> </DIV><DIV>Consul logs specifies "==> bind_addr cannot be empty" I have added "<SPAN>consul_bind_adapter = eth0 </SPAN>" in inventory.ini file.</DIV> Tue, 14 May 2019 14:07:25 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558630#M16254 stalasani 2019-05-14T14:07:25Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558632#M16255 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/101017">@stalasani</a> ,</P> <P> </P> <P>I would like to review /etc/sysconfig/sas/sas-viya-consul-default from each machine. Also, show me the output from a command below:</P> <P> </P> <PRE><CODE class=" language-sas">ansible all -m shell -a "ifconfig"</CODE></PRE> Tue, 14 May 2019 14:22:12 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558632#M16255 alexal 2019-05-14T14:22:12Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558649#M16256 <P><SPAN>/etc/sysconfig/sas/sas-viya-consul-default info from each machine.</SPAN></P><P> </P><P>363748lp42sg001.geicoddc.net</P><P> </P><P># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL='eth0'</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"<BR />export CONSUL_BOOTSTRAP_EXPECT='3'<BR />export CONSUL_DATACENTER_NAME='sas-viya-pd-03may2019'<BR />export CONSUL_SERVER_FLAG='true'<BR />export CONSUL_SERVER_LIST='10.190.103.251, 10.190.103.210, 10.190.103.238'<BR />export DISABLE_CONSUL_HTTP_PORT='True'<BR />export SECURE_CONSUL='True'</P><P>363748lp42sg002.geicoddc.net</P><P><BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL='eth0'</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"<BR />export CONSUL_BOOTSTRAP_EXPECT='3'<BR />export CONSUL_DATACENTER_NAME='sas-viya-pd-03may2019'<BR />export CONSUL_SERVER_FLAG='true'<BR />export CONSUL_SERVER_LIST='10.190.103.251, 10.190.103.210, 10.190.103.238'<BR />export DISABLE_CONSUL_HTTP_PORT='True'<BR />export SECURE_CONSUL='True'</P><P>363748lp42sg003.geicoddc.net</P><P><BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL='eth0'</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"<BR />export CONSUL_BOOTSTRAP_EXPECT='3'<BR />export CONSUL_DATACENTER_NAME='sas-viya-pd-03may2019'<BR />export CONSUL_SERVER_FLAG='true'<BR />export CONSUL_SERVER_LIST='10.190.103.251, 10.190.103.210, 10.190.103.238'<BR />export DISABLE_CONSUL_HTTP_PORT='True'<BR />export SECURE_CONSUL='True'</P><P>363748lp42sg004.geicoddc.net</P><P><BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"<BR />363748lp42sg005.geicoddc.net<BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"</P><P>363748lp42sg006.geicoddc.net<BR /></P><P># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"</P><P>363748lp42sg007.geicoddc.net</P><P><BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"<BR />363748lp42sg008.geicoddc.net<BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"</P><P>363748vp42sg001.geicoddc.net</P><P><BR /># Consul option: -advertise<BR /># Specify the name of a network interface or IPv4 address.<BR /># Defaults to the bind IP address if not specified.<BR />#CONSUL_ADVERTISE_INTERNAL=</P><P># Consul option: -bind<BR /># Holds the desired name of a network interface or IPv4 address.<BR /># Please do not edit this. Instead, set it via your Ansible run.<BR />export CONSUL_BIND_EXTERNAL=</P><P>CONSUL_CONFIG_DIR="/opt/sas/viya/config/etc/consul.d"<BR />CONSUL_OPTIONS="-client 0.0.0.0"<BR />CONSUL_URL="http://localhost:8500"</P> Tue, 14 May 2019 15:08:51 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558649#M16256 stalasani 2019-05-14T15:08:51Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558652#M16257 <P><SPAN>/etc/sysconfig/sas/sas-viya-consul-default and ansible output.</SPAN></P> Tue, 14 May 2019 15:14:22 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558652#M16257 stalasani 2019-05-14T15:14:22Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558731#M16261 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/101017">@stalasani</a> ,</P> <P> </P> <P>In the inventory.ini file, you did set consul_bind_adapter=eth0 for each of your hosts, but the network adapter with name eth0 exist only on Ansible_Controller. You have to set a proper network adapter for each of your hosts in the inventory file.</P> Tue, 14 May 2019 18:47:02 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/558731#M16261 alexal 2019-05-14T18:47:02Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559753#M16285 <P>This error has been fixed but the deployment failed sasrabbitmq user error.</P><P>fatal: [Service_Layer_I]: FAILED! => {"changed": true, "cmd": ["/opt/sas/viya/home/bin/setup_rabbit_cluster", "--hostlist", "363748lp42sg001.geicoddc.net, 363748lp42sg002.geicoddc.net, 363748lp42sg003.geicoddc.net", "--home", "/opt/sas/viya/home", "--config", "/opt/sas/viya/config", "--service", "sas-viya-rabbitmq-server-default", "--logfile", "/tmp/sas_setup_rabbit_cluster.log"], "delta": "0:00:02.064688", "end": "2019-05-14 15:23:32.091912", "msg": "non-zero return code", "rc": 3, "start": "2019-05-14 15:23:30.027224", "stderr": "chown: invalid user: ‘sasrabbitmq’", "stderr_lines": ["chown: invalid user: ‘sasrabbitmq’"], "stdout": "Tue May 14 15:23:30 EDT 2019 setup_rabbit_cluster *************\nTue May 14 15:23:30 EDT 2019 setup_rabbit_cluster Beginning SAS RabbitMQ clustering setup\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster SSL is true\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster CHECK_PORT is 5671\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster First host in list is 363748lp42sg001.geicoddc.net\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster First host shortname is 363748lp42sg001\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster Primary Host is 363748lp42sg001.geicoddc.net\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster Primary Short Host is 363748lp42sg001\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster My Host is 363748lp42sg001.geicoddc.net\nTue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster I am the primary host\nTue May 14 15:23:30 EDT 2019 setup_rabbit_cluster Copy generated Erlang shared secret to Rabbit.\nTue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Copying generated Erlang shared secret to Rabbit.\nTue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Could not change ownership of generated shared secret file /opt/sas/viya/config/var/lib/rabbitmq-server/sasrabbitmq/.erlang.cookie\nTue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Target owner is sasrabbitmq\nTue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Returned status was 1\nTue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Setup failed. Beware of independent Rabbit hosts running with different shared secrets.", "stdout_lines": ["Tue May 14 15:23:30 EDT 2019 setup_rabbit_cluster *************", "Tue May 14 15:23:30 EDT 2019 setup_rabbit_cluster Beginning SAS RabbitMQ clustering setup", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster SSL is true", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster CHECK_PORT is 5671", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster First host in list is 363748lp42sg001.geicoddc.net", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster First host shortname is 363748lp42sg001", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster Primary Host is 363748lp42sg001.geicoddc.net", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster Primary Short Host is 363748lp42sg001", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster My Host is 363748lp42sg001.geicoddc.net", "Tue May 14 15:23:30 EDT 2019 (debug) setup_rabbit_cluster I am the primary host", "Tue May 14 15:23:30 EDT 2019 setup_rabbit_cluster Copy generated Erlang shared secret to Rabbit.", "Tue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Copying generated Erlang shared secret to Rabbit.", "Tue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Could not change ownership of generated shared secret file /opt/sas/viya/config/var/lib/rabbitmq-server/sasrabbitmq/.erlang.cookie", "Tue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Target owner is sasrabbitmq", "Tue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Returned status was 1", "Tue May 14 15:23:32 EDT 2019 setup_rabbit_cluster Setup failed. Beware of independent Rabbit hosts running with different shared secrets."]</P><P> </P><P>I created local account "sasrabbitmq" on rabbitmq servers and added it to be part of sas group.</P><P> </P><P>Now the deployment fails with below error.</P><P>2019-05-17 11:44:44,337 p=20286 u=root | fatal: [Service_Layer_III]: FAILED! => {"changed": true, "cmd": ["/opt/sas/viya/home/bin/setup_rabbit_cluster", "--hostlist", "363748lp42sg001.geicoddc.net, 363748lp42sg002.geicoddc.net, 363748lp42sg003.geicoddc.net", "--home", "/opt/sas/viya/home", "--config", "/opt/sas/viya/config", "--service", "sas-viya-rabbitmq-server-default", "--logfile", "/tmp/sas_setup_rabbit_cluster.log"], "delta": "0:00:26.460912", "end": "2019-05-17 11:44:44.285005", "msg": "non-zero return code", "rc": 8, "start": "2019-05-17 11:44:17.824093", "stderr": "Error: unable to perform an operation on node 'rabbit@363748lp42sg001.geicoddc.net'. Please see diagnostics information and suggestions below.\n\nMost common reasons for this are:\n\n * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)\n * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)\n * Target node is not running\n\nIn addition to the diagnostics info below:\n\n * See the CLI, clustering and networking guides on <A href="http://rabbitmq.com/documentation.html" target="_blank">http://rabbitmq.com/documentation.html</A> to learn more\n * Consult server logs on node rabbit@363748lp42sg001.geicoddc.net\n\nDIAGNOSTICS\n===========\n\nattempted to contact: ['rabbit@363748lp42sg001.geicoddc.net']\n\nrabbit@363748lp42sg001.geicoddc.net:\n * connected to epmd (port 4369) on 363748lp42sg001.geicoddc.net\n * epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traffic \n * TCP connection succeeded but Erlang distribution failed \n\n * Authentication failed (rejected by the remote node), please check the Erlang cookie\n\n\nCurrent node details:\n * node name: 'rabbitmqcli17@363748lp42sg003.geicoddc.net'\n * effective user's home directory: /home/sasrabbitmq\n * Erlang cookie hash: DC2DocmZ/sdjb8gZkaqU8w==", "stderr_lines": ["Error: unable to perform an operation on node 'rabbit@363748lp42sg001.geicoddc.net'. Please see diagnostics information and suggestions below.", "", "Most common reasons for this are:", "", " * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)", " * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)", " * Target node is not running", "", "In addition to the diagnostics info below:", "", " * See the CLI, clustering and networking guides on <A href="http://rabbitmq.com/documentation.html" target="_blank">http://rabbitmq.com/documentation.html</A> to learn more", " * Consult server logs on node rabbit@363748lp42sg001.geicoddc.net", "", "DIAGNOSTICS", "===========", "", "attempted to contact: ['rabbit@363748lp42sg001.geicoddc.net']", "", "rabbit@363748lp42sg001.geicoddc.net:", " * connected to epmd (port 4369) on 363748lp42sg001.geicoddc.net", " * epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traffic ", " * TCP connection succeeded but Erlang distribution failed ", "", " * Authentication failed (rejected by the remote node), please check the Erlang cookie", "", "", "Current node details:", " * node name: 'rabbitmqcli17@363748lp42sg003.geicoddc.net'", " * effective user's home directory: /home/sasrabbitmq", " * Erlang cookie hash: DC2DocmZ/sdjb8gZkaqU8w=="], "stdout": "Fri May 17 11:44:17 EDT 2019 setup_rabbit_cluster *************\nFri May 17 11:44:17 EDT 2019 setup_rabbit_cluster Beginning SAS RabbitMQ clustering setup\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster SSL is true\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster CHECK_PORT is 5671\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster First host in list is 363748lp42sg001.geicoddc.net\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster First host shortname is 363748lp42sg001\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster Primary Host is 363748lp42sg001.geicoddc.net\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster Primary Short Host is 363748lp42sg001\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster My Host is 363748lp42sg003.geicoddc.net\nFri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster I am a clustering host\nFri May 17 11:44:18 EDT 2019 setup_rabbit_cluster Copy generated Erlang shared secret to Rabbit.\nFri May 17 11:44:22 EDT 2019 setup_rabbit_cluster Copying generated Erlang shared secret to Rabbit.\nFri May 17 11:44:22 EDT 2019 setup_rabbit_cluster Starting sas-viya-rabbitmq-server-default\nFri May 17 11:44:37 EDT 2019 setup_rabbit_cluster Wait for a Rabbit listener instance on the primary host 363748lp42sg001.geicoddc.net\nFri May 17 11:44:37 EDT 2019 (debug) setup_rabbit_cluster wait_for_host 363748lp42sg001.geicoddc.net succeeded, listener detected.\nFri May 17 11:44:39 EDT 2019 (debug) setup_rabbit_cluster Issuing clustering commands\nFri May 17 11:44:39 EDT 2019 (debug) setup_rabbit_cluster Stop the Rabbit app (not the server though)\nFri May 17 11:44:41 EDT 2019 (debug) setup_rabbit_cluster USE_LONGNAME = true, MY_PRIMARY is rabbit@363748lp42sg001.geicoddc.net\nFri May 17 11:44:41 EDT 2019 (debug) setup_rabbit_cluster Cluster with the primary node\nFri May 17 11:44:44 EDT 2019 setup_rabbit_cluster Attempts to join_cluster with the node rabbit@363748lp42sg001.geicoddc.net failed. Clustering failed.\nFri May 17 11:44:44 EDT 2019 setup_rabbit_cluster Returned status was 69", "stdout_lines": ["Fri May 17 11:44:17 EDT 2019 setup_rabbit_cluster *************", "Fri May 17 11:44:17 EDT 2019 setup_rabbit_cluster Beginning SAS RabbitMQ clustering setup", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster SSL is true", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster CHECK_PORT is 5671", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster First host in list is 363748lp42sg001.geicoddc.net", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster First host shortname is 363748lp42sg001", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster Primary Host is 363748lp42sg001.geicoddc.net", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster Primary Short Host is 363748lp42sg001", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster My Host is 363748lp42sg003.geicoddc.net", "Fri May 17 11:44:18 EDT 2019 (debug) setup_rabbit_cluster I am a clustering host", "Fri May 17 11:44:18 EDT 2019 setup_rabbit_cluster Copy generated Erlang shared secret to Rabbit.", "Fri May 17 11:44:22 EDT 2019 setup_rabbit_cluster Copying generated Erlang shared secret to Rabbit.", "Fri May 17 11:44:22 EDT 2019 setup_rabbit_cluster Starting sas-viya-rabbitmq-server-default", "Fri May 17 11:44:37 EDT 2019 setup_rabbit_cluster Wait for a Rabbit listener instance on the primary host 363748lp42sg001.geicoddc.net", "Fri May 17 11:44:37 EDT 2019 (debug) setup_rabbit_cluster wait_for_host 363748lp42sg001.geicoddc.net succeeded, listener detected.", "Fri May 17 11:44:39 EDT 2019 (debug) setup_rabbit_cluster Issuing clustering commands", "Fri May 17 11:44:39 EDT 2019 (debug) setup_rabbit_cluster Stop the Rabbit app (not the server though)", "Fri May 17 11:44:41 EDT 2019 (debug) setup_rabbit_cluster USE_LONGNAME = true, MY_PRIMARY is rabbit@363748lp42sg001.geicoddc.net", "Fri May 17 11:44:41 EDT 2019 (debug) setup_rabbit_cluster Cluster with the primary node", "Fri May 17 11:44:44 EDT 2019 setup_rabbit_cluster Attempts to join_cluster with the node rabbit@363748lp42sg001.geicoddc.net failed. Clustering failed.", "Fri May 17 11:44:44 EDT 2019 setup_rabbit_cluster Returned status was 69"]}</P> Fri, 17 May 2019 18:42:43 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559753#M16285 stalasani 2019-05-17T18:42:43Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559757#M16286 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/101017">@stalasani</a> ,</P> <BLOCKQUOTE> <P>I created local account "sasrabbitmq" on rabbitmq servers and added it to be part of sas group.</P> </BLOCKQUOTE> <P>That user should be created automatically during the deployment process. Anyway, what home directory did you set for that user? The home directory for that user should be /opt/sas/viya/config/var/lib/rabbitmq-server/sasrabbitmq and the user should have read/write access to it in order to write Erlang cookie.</P> Fri, 17 May 2019 18:57:01 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559757#M16286 alexal 2019-05-17T18:57:01Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559761#M16287 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/41748">@alexal</a> </P><P> </P><P>Deployment didn’t create the account. That is the reason it failed during initial deployment.</P><P> </P><P>chown: invalid user: ‘sasrabbitmq’", "stderr_lines": ["chown: invalid user: ‘sasrabbitmq’"], "stdout": "Tue May 14 15:23:30 EDT 2019 setup_rabbit_cluster *************\nTue</P><P> </P><P>I didn’t specify any home directory, should I update the home directory and re-deploy?</P><P> </P><P>Thank you!</P> Fri, 17 May 2019 19:07:59 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/559761#M16287 stalasani 2019-05-17T19:07:59Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560109#M16305 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/101017">@stalasani</a> ,</P> <BLOCKQUOTE> <P><SPAN>I didn’t specify any home directory, should I update the home directory and re-deploy?</SPAN></P> </BLOCKQUOTE> <P>Yes.</P> Mon, 20 May 2019 12:24:30 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560109#M16305 alexal 2019-05-20T12:24:30Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560163#M16306 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/41748">@alexal</a> </P><P> </P><P>Thank you for the confirmation. I modifying the home directory for "sarabbitmq" on last Friday and was able to successfully deploy SAS Viya without any issue.</P><P> </P><P>I'll be restarting the services on all nodes to makes sure the system is healthy. I'll keep you posted on any updated.</P><P> </P><P>Thank you,</P><P>Sukesh Talasani</P> Mon, 20 May 2019 14:29:16 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560163#M16306 stalasani 2019-05-20T14:29:16Z Re: SAS Viya deployment error in cosul logs "==> bind_addr cannot be empty" https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560164#M16307 <P><a href="https://communities.sas.com/t5/user/viewprofilepage/user-id/101017">@stalasani</a> ,</P> <P><BR />Thanks for the update. I'm glad that all the issues were resolved. Please do not forget to update the track and mark this thread as resolved.</P> Mon, 20 May 2019 14:31:29 GMT https://communities.sas.com/t5/Administration-and-Deployment/SAS-Viya-deployment-error-in-cosul-logs-quot-gt-bind-addr-cannot/m-p/560164#M16307 alexal 2019-05-20T14:31:29Z