Hi!
I need to set up a test environment as similar as possible to current live system. So my approach was to clone the current quite new system (to avoid all the configuration from zero), replace hostnames (+certificates) and changing passwords and cleaning everything up from real data.
* cloning - success
* replacing hostnames - success
* replacing passwords - success (with wizard + manually for cpmowner and cpmuser - not mentioned in manuals, got 404 when starting CPM as it couldn't log to database)
* replaced certificates - half success (because of java keystorage)
* cleaning from real data - half success (because of auto backups)
When cleaning from real data i also need to clear the data from automatic backups (from SASApp and SASApp_VA) stored at config/Lev(n)/Backup/Vault/. As i found out the only way to do that is to set the automatic backups as short as needed (for a day for example) and then it removes those backups by itself. As i understand there are two ways to modify backup- from Management Console or from Enironment Manager. Either way the connection between Environment Manager server (hq-server) and agent (hq-agent) must work or the settings won't reach to SASApp and SASApp_VA servers.
Now the problem is - as the hostnames and passwords has been changed - that Environment Manager connections between hq-server and hq-agents do not work. So to update the certificates (the date for config/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore file was the same as installation for the original setup, this means the Deployment Manager wizard is not updating all the certificates - should it?) i followed the manual <https://documentation.sas.com/?docsetId=bimtag&docsetTarget=p1fpnnm9hxkhlzn1x5tkqs1caeg5.htm&docsetV... steps "Generate the SAS Environment Manager Keystore with a Self-Signed Certificate" and "Update Certificates for SAS Environment Manager". After several tries it seems that the hq-server at least starts up when i won't delete eam_keystore entries (previous link, title "Delete the Keystore From the Database"). So this means that it still use some records from old setup and probably maybe even password and/or certificates or some salt from those. Or i still haven't replaced all the certificates/passwords.
Anyway, even with this old eam_keystore entry the hq-server eventually starts up and the webpage (7443) asks for credentials. I still get "SAS Environment Manager Login Error" but i could live with that at the moment as i need to set up the connection between hq-server and hq-agents to Apply automatic backup settings to the agents.
After flushing the data/ and log/ directory contents and staring the agent:
Starting HQ Agent...... running (15079).
[ Running agent setup ]
Should Agent communications to HQ be unidirectional [default=no]: yes
What is the HQ server IP address: <hostname>
Should Agent communications to HQ always be secure [default=yes]: Yes
What is the HQ server SSL port [default=7443]: 7443
- Testing secure connection ... Success
What is your HQ login [default=hqadmin]: sasevs@saspw
What is your HQ password: **Not echoing value**
What is the agent IP address [default=x.x.x.x]: <hostname>
- Received temporary auth token from agent
- Registering agent with HQ
- Unable to register agent: Permission denied
and log:
21-09-2020 13:00:09,683 EEST ERROR [autoinventory-scanner] [AutoinventoryCommandsServer@280] Unable to send autoinventory platform data to server, sleeping for 75 secs before retrying. Error: Unable to communicate with server -- provider not yet setup
and hq-server's server.log:
21-09-2020 12:57:46,354 EEST INFO [tomcat-http--30] [com.sas.hyperic.security.CasIdentityRetrievingTicketValidator@61] The logon username return by SSO, before getting current user. is: sasevs@saspw
21-09-2020 12:57:46,356 EEST INFO [tomcat-http--30] [org.jasig.cas.client.proxy.ProxyGrantingTicketStorageImpl@76] No Proxy Ticket found for [].
21-09-2020 12:57:46,359 EEST INFO [tomcat-http--30] [com.sas.hyperic.security.SasSimpleUserDetailsService@97] Userdetailservice provider is:com.sas.hyperic.security.sasserver.SASIdentityServiceProvider@575c6304
21-09-2020 12:57:46,360 EEST INFO [tomcat-http--30] [com.sas.hyperic.security.SasSimpleUserDetailsService@110] User [sasevs@saspw] does not exist in EVM local database.
21-09-2020 12:57:46,360 EEST ERROR [tomcat-http--30] [com.sas.hyperic.security.SasSimpleUserDetailsService@136] java.lang.NullPointerException
21-09-2020 12:57:46,375 EEST INFO [tomcat-http--30] [org.hyperic.hq.ui.security.BaseSessionInitializationStrategy@180] User subject is null, don't auto-create user, program will return without executing afterward logic.
So my first problem is to get rid of those SASApp and SASApp_VA backups located in Vault directory. Maybe i could just remove those with rm? What metadata thinks about this?
And the second goal would be to get up the connection between Environment Manager server and agents.
I'm i missing something? I know my weak spot could be generating certificates, especially Java keystore ones but maybe there's some hint i just haven't come to.
Yes, the new jks certificate is generated from 1024b RSA key. For testing purposes hq-server.conf server.keystore.password parameter is plain text (to avoid ENC() encoded password miscofiguration. By the way- how do i encrypt the password to ENC()? Tried like described https://docs.oracle.com/cd/E35976_01/server.740/es_admin/src/tadm_ssl_convert_pem_to_jks.html).
Thanks!
PL
I got the hq-server up and running after following the step-by-step how-to i got form our local support guy.
The problem was badly made keystore certificates, probably. I'll paste it here also, maybe helps someone.
Please follow the steps below to update the SAS Environment Manager certificate (hyperic.keystore)
<SAS_CONFIG>/Lev2/Web/SASEnvironmentManager/server-5.8.0-EE/bin/hq-server.sh stop
<SAS_CONFIG>/Lev2/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/hq-agent.sh stop
mv <SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore <SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore.bkup
cd <SASHome>\SASWebServer\9.4\httpd-2.X.X.X-64\bin>
openssl pkcs12 -export -chain -inkey <saswebserver.key> -in <saswebserver.crt> -name hq -CAfile <Chaincerts.txt> -out /tmp/hyperic.p12
Note: saswebserver.key , saswebserver.crt and Chaincerts.txt need to be replaced with the actual files which represent your SAS Web Server Key, SAS Web Server certificate and chain(if it exists) respectively
Chaincerts.txt – is a file created by concatenating the contents of the root, intermediate and server certificates in a single file
When prompted for a password, enter hyperic
cd <SASHome> \ SASPrivateJavaRuntimeEnvironment \ 9.4 \ jre \ bin>
keytool -importkeystore -deststorepass hyperic -destkeypass hyperic -destkeystore <SASConfig>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore -srckeystore /tmp/hyperic.p12 -srcstoretype PKCS12 -srcstorepass hyperic -alias hq
Note: SASConfig needs to be replaced with the actual path to the configuration folder.
Ignore any warning message that you get, so long as the hyperic.keystore file is generated correctly , you are good
POSTGRES_HOME=<SAS_HOME>/SASHome/SASWebInfrastructurePlatformDataServer/9.4
export PATH=${POSTGRES_HOME}/bin:$PATH
export LD_LIBRARY_PATH=${POSTGRES_HOME}/lib:$LD_LIBRARY_PATH
where SAS_HOME is the actual folder path of SAS Installation Folder.
Run the following PSQL command
That table should list one entry/row.
If that row exists, then issue the following psql command to delete that row:
<SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/bin/hq-server.sh start
<SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/hq-agent.sh start
Try replacing the data directory of EVM agent and then restarting it.
mv SAS-configuration-directory/Web/SASEnvironmentManager/agent-version-EE/data SAS-configuration-directory/Web/SASEnvironmentManager/agent-version-EE/data.old
Thanks for the reply! I have flushed everything from data/ and log/ after every restart of EVM agent. Still no luck.
Was there any alias or load balancer used for web applications in the initial configuration?
If yes, you will have to update those entries manually.
For hyperic.keystore, yes it has to be generated again with new certificates. I think the steps are documented in the guide you have shared.
As part of updating hostnames there is a file generated for some manual changes which has some steps to be done in the agent configuration, have you performed them?
There is a manual step on the compute server where EVM datamart is hosted to change the hostname manually in the sasev.properties file. Have you performed that as well?
I had a similar issue after cloning of environment and it got resolved after all the required manual steps and flushing data folders.
I got the hq-server up and running after following the step-by-step how-to i got form our local support guy.
The problem was badly made keystore certificates, probably. I'll paste it here also, maybe helps someone.
Please follow the steps below to update the SAS Environment Manager certificate (hyperic.keystore)
<SAS_CONFIG>/Lev2/Web/SASEnvironmentManager/server-5.8.0-EE/bin/hq-server.sh stop
<SAS_CONFIG>/Lev2/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/hq-agent.sh stop
mv <SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore <SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore.bkup
cd <SASHome>\SASWebServer\9.4\httpd-2.X.X.X-64\bin>
openssl pkcs12 -export -chain -inkey <saswebserver.key> -in <saswebserver.crt> -name hq -CAfile <Chaincerts.txt> -out /tmp/hyperic.p12
Note: saswebserver.key , saswebserver.crt and Chaincerts.txt need to be replaced with the actual files which represent your SAS Web Server Key, SAS Web Server certificate and chain(if it exists) respectively
Chaincerts.txt – is a file created by concatenating the contents of the root, intermediate and server certificates in a single file
When prompted for a password, enter hyperic
cd <SASHome> \ SASPrivateJavaRuntimeEnvironment \ 9.4 \ jre \ bin>
keytool -importkeystore -deststorepass hyperic -destkeypass hyperic -destkeystore <SASConfig>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/conf/hyperic.keystore -srckeystore /tmp/hyperic.p12 -srcstoretype PKCS12 -srcstorepass hyperic -alias hq
Note: SASConfig needs to be replaced with the actual path to the configuration folder.
Ignore any warning message that you get, so long as the hyperic.keystore file is generated correctly , you are good
POSTGRES_HOME=<SAS_HOME>/SASHome/SASWebInfrastructurePlatformDataServer/9.4
export PATH=${POSTGRES_HOME}/bin:$PATH
export LD_LIBRARY_PATH=${POSTGRES_HOME}/lib:$LD_LIBRARY_PATH
where SAS_HOME is the actual folder path of SAS Installation Folder.
Run the following PSQL command
That table should list one entry/row.
If that row exists, then issue the following psql command to delete that row:
<SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/server-5.8.0-EE/bin/hq-server.sh start
<SAS_CONFIG>/Lev1/Web/SASEnvironmentManager/agent-5.8.0-EE/bin/hq-agent.sh start
The SAS Users Group for Administrators (SUGA) is open to all SAS administrators and architects who install, update, manage or maintain a SAS deployment.
SAS technical trainer Erin Winters shows you how to explore assets, create new data discovery agents, schedule data discovery agents, and much more.
Find more tutorials on the SAS Users YouTube channel.