BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
pedrammobedi
Quartz | Level 8

As of late, the performance of our Visual Analytics have significantly slowed down. This includes logging in, browsing and loading reports. I know there can be a lot of reasons for this. But I do not remember modifying any settings. Could this be an issue with the server computer?

 

Anyhow, is there a check list to go through to detect the cause?

1 ACCEPTED SOLUTION

Accepted Solutions
JuanS_OCS
Amethyst | Level 16

Hey there,

 

sorry, what i need is the wrapper.conf under /Config/Lev1/Web/WebAppServer/SASServer1_1/conf. You should find some lines with information such as:

wrapper.java.additional.7=-Xmx4096m
wrapper.java.additional.8=-Xss256k
wrapper.java.additional.9=-Xms1024m
wrapper.java.additional.10=-XX:PermSize=768m
wrapper.java.additional.11=-XX:MaxPermSize=1280m

How many cores do you have on the server (logical or virtual), which speed and how much RAM memory?

 

Now, and as heads up of what my advise will be based on:

 

SAS recommends (by default) on Single Web App Server ( http://support.sas.com/documentation/cdl/en/appsrvtuning/68923/HTML/default/viewer.htm#n1qc1gus8n8a5... ) the following values. But you should extend them until the performance is good again. I have on some systems values of 20480m for Xms and Xmx and 2048 for PermSize and MaxPermSize. You should start with the recommended values, and then start to make them grow. After each change, you will need to restart the SASServer1_1 service (do not forget to clean or achive and clean SASServer1_1/tmp, SASServer1_1/work and SASServer1_1/logs after stoping and before starting the SASserver1_1:

Large
-Xms
-Xmx
-XX:PermSize
-XX:MaxPermSize
8192m
8192m
1664m
1664m

 

Large
-Dsas.svcs.http.max.total.connections
512
-Dsas.svcs.http.max.connections
512

 

SharedServices JDBC Resource 

Large
maxPoolSize
256

 

Tomcat Executor Thread Pool

Large
maxThreads
1024

 

 

More things you can do to tune your system:

 

Operating System: http://support.sas.com/documentation/cdl/en/appsrvtuning/68923/HTML/default/viewer.htm#n0xfno04p3q3u...

 

And those other two, which can be found under: http://support.sas.com/documentation/cdl/en/appsrvtuning/68923/HTML/default/viewer.htm#n11d31ckagwav...

 

PS. What @userid007 whanted to mention is the SAS Environment Manager metrics: http://support.sas.com/documentation/cdl/en/appsrvtuning/68923/HTML/default/viewer.htm#p1kwvsj8oujet...

 

 

 

View solution in original post

30 REPLIES 30
JuanS_OCS
Amethyst | Level 16

This seems a performance problem focused on the middle tier. I wonder if the number of your users/concurrent users has increased lately, of if the server (specially middle tier) has not been restarted for some time.

 

1- I would check if the initial sizing pre-requisites of the VA server has changed. If yes, you should contact your SAS representative. If no, please continue with this little list.

2- I would restart the middle tier services (Web Server, Web Application Servers, Cache Locator, JMS, etc) and archive/clean the termporary files such as logs and the /work and /tmp folders fo the Web Application Servers.

3- If this still is not solving your problem, there is a guide to increase the performance of your middle tier:https://support.sas.com/documentation/cdl/en/appsrvtuning/68923/PDF/default/appsrvtuning.pdf

4- You can always contact SAS Technical Support to advise you on the best way to move forward and keep track of your challenges and achievements.

pedrammobedi
Quartz | Level 8

Thank you!

I will try the first 3 points you mention and come back to you.

JuanS_OCS
Amethyst | Level 16

Perfect, thank you 🙂 Looking forward for the outcome of your checks/

 

 

 

pedrammobedi
Quartz | Level 8

Hey! The restart process of my webserver is still running and it has taken much longer than expected. It took about 15 minutes before.

Here I give you the size of my logs, temp and work folders.

  1. logs: 13.3 GB
  2. temp: 31.2 MB
  3. work: 111 MB

Do any of these look unusual? I think the logs folder has become quite bulky. But is there a systematic way of cleaning it or should I just empty the text files manually?

pedrammobedi
Quartz | Level 8

I have restarted all the services and the web app server. Nothing much has changed. I don't honestly dare approaching the settings mentioned in the Tuning pdf you provided but I am almost sure no one has been here for a while to change them.

LinusH
Tourmaline | Level 20

I have no specific experience with VA/LASR logs, but 13 GB sounds quite large.

You may need to review your logging settings.

Also check the logs. There can be a lot of new types of warnings/errors that can be relevant in your investigation. I should be quite easy to see if the logging has elevated at the same time as your performance problems started.

Data never sleeps
pedrammobedi
Quartz | Level 8

There are many log files. Whch ones in specific should I check?

Also please take a look at the reply I posted about Java(TM) Platform SE Binary. Thanks!

pedrammobedi
Quartz | Level 8

I see that Java Platform SE Binary is in average consuming 30-40% of my CPU. It's an active process even when nobody is using SAS VA. Could this be a reason?

JuanS_OCS
Amethyst | Level 16

Hi,

 

1- as @LinusH said well, those big files on logs, even when no user is there and you already cleaned the logs, it seems to indicate that some of your log configurations are on too high detail level: http://support.sas.com/documentation/cdl/en/bimtag/68217/HTML/default/viewer.htm#p0qy7oulxf2ot7n1bca...

 

You might want to change some levels from debug to error or info, what it was default.

 

2- can you get the full command line of the executable Java Platform SE Binary that consumes those resources? 

  a) this can help ups to locate and identify the process that might be an issue.

  b) 

 

3- I believe to understand your VA installation is on Windows, in that case is non-distributed. How is your deployment looking like? All installed in 1 server? Metadata/Compute/Middle-tier separated? 

 

4- Do you use the AutoLoad functionality or only Queries/Schemas?

 

5- Is it enabled recently the Performance Reports/ Environment Manager reports/APM/etc?

userid007
Obsidian | Level 7

as @JuanS_OCS mentioned, the critical issue would be the autoload logs under VA<config>Levn\Applications\SASVisualAnalytics\VisualAnalyticsAdministrator

 

I had to setup a clean up job to remove all the small log files as the number of files were huge. 

 

perhaps cleanup your logs directory and if you are in VM's check if there is any cpu wait for your VM.

Sai Korrapati
pedrammobedi
Quartz | Level 8

I have tried this to no avail. 

JuanS_OCS
Amethyst | Level 16

Hi @pedrammobedi, since some days passed from your original question, I feel curious if you could make progress with your performance issues on VA. Any feedback will be welcome!

 

Best regards,

Juan

pedrammobedi
Quartz | Level 8

Hi! Thank you for your response. Unfortunately the problem lingers. And I got more help from you people in the community that I received from SAS. They have simply directed me to learning Environment Manager in which I still do not see anything suspicious. All I know is that Java Platform SE binary is taking a lot of CPU even when nobody is using SAS. However, it is still hard to say what is causing it to work that hard. The only thing I see is my SAS VA taking a very long time to load. So any help is still appreciated. 

JuanS_OCS
Amethyst | Level 16

Hi,

 

sorry to learn that you still have issues there, but let's see if we can do something about it.

 

On one hand, I would keep open a track with SAS Technical Support and let them know that Environment Manager is not helping this time. It is a good tool, but not in your case for now.

 

In the other side, since most processes in SAS (metadata, compute and web tiers) are java processes, first we would need more information about which specific SAS process is giving you the high level of CPU usage, in order to filter from the huge amount of options available where the performance problem may came from.

 

For that, if you server is Lunux, ps -ef | grep java would help.

If your server is Windows, you can go to the Task Manager, and on the details tab, you can add the Command Line column and try to extend it as much as possible to get the full command.

 

Something else that may help is to know:

 

- Do you use AutoLoad or Data Builder Queries ?

- How your deployment looks like? How many servers compose the SAS VA application?

 

 

Ready to join fellow brilliant minds for the SAS Hackathon?

Build your skills. Make connections. Enjoy creative freedom. Maybe change the world. Registration is now open through August 30th. Visit the SAS Hackathon homepage.

Register today!
Tips for filtering data sources in SAS Visual Analytics

See how to use one filter for multiple data sources by mapping your data from SAS’ Alexandria McCall.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 30 replies
  • 9704 views
  • 4 likes
  • 7 in conversation