SAS 9.4 SMC Unresponsive when checking Libaray Property Dataserver tab

[Prab]

Hi All,

We have been facing an issue when checking oracle library properties--->Data Server Tab from SAS 9.4 SMC.

Issue is whole SAS 9.4 SMC is unresponsive for 2 mins and we cant see any items under Server Details and Connection details. It was displaying blank.

I only found below Message from Metadata Server log:

2016-02-27T10:53:14,370 INFO  [57074050] 6456979:XXXXXXX@saspw - GetMetadataObjects(0x218c) OBJ=1505708 RQ=1416 RESP=768370408 TIME=61.157 TYPE="ServerComponent" SEARCH="*[@ClassIdentifier='ORACLE']"

PFA screenshot. Please help us??

Thanks,

Prabhu

[LinusH]

Does the libname work using the Metadata registration from lets say EG?
Perhaps the meradata is corrupt. What has changed in the environment? I assume that this has worked before, yes?
Either restore Metadata from a point where you know this was working or delete the registration and start over.

[Prab]

Thank you LinusH.

I will get back to you on regsitering from SAS EG.

I have tried to create library from SMC but again we found unresponsive SMC issue

Yes we had tried from SAS DI studio to see Data Server properties...We didnt have any issues here in SAS DI.

This is not only for single table/library issue..Issue exists across Data Library Manager -Libraries.

Important point is we dont have issue in DEV and INT environments...I mean no SMC unresposive while checking Data Server tab.

Actually,Sorry, we don know last working date stamp..We found this issue by ourself..

Not sure what went wrong here...

Thank you for your support.

Prab

[SASKiwi]

Also try creating a link to the same data source in a LIBNAME statement that doesn't use metadata. Does that perform OK?

[Prab]

Sorry..I didnt get you..Can you eloborate..Thank you.

[SASKiwi]

Write and run a LIBNAME statement like this and test if performance is good or not. If it is good then the problem is with your metadata. If it is bad then the problem is with your Oracle connection not metadata.

libname test oracle path='MyPath' user=MyUser password='MyPassword';

[LinusH]

Given the information the same Metadata is accessible from DI studio. So metadata seems fine.
I guess that you already have tried SMC different clients?
A part from regular advice such as restarting the server, reinstalling SMC and potentially migrate/promote Metadata from an environment that works - I have no clue. Might be best to contact SAS tech support.

[Prab]

Yes, We have tried SMC in different clients.Thank you for all your comments.

[JuanS_OCS]

Hi,

A question: did you already tried to enebke the logging on the SMC?

If not, I would recommend it, to get additional information:
http://support.sas.com/kb/43/157.html

[Prab]

Hi, Yes this was enabled. Please see below trace after enabling option for SMC Logging.

setting permissions on /home/giecinsp/.SASAppData...
read permission denied for everyone
write permission denied for everyone
execute permission denied for everyone
read permission successfully set for owner only
write permission successfully set for owner only
execute permission successfully set for owner only
       1|INFO   |e.folderview.visuals|Log                    |initLogging : Logging initialized, level = INFO
workspacepath=/home/biecinsp/.SASAppData/MetadataServerProfiles
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:57)
        at java.nio.CharBuffer.allocate(CharBuffer.java:331)
        at com.sas.iom.orb.brg.Marshaller.decodeString(Marshaller.java:3261)
        at com.sas.iom.orb.brg.Marshaller.unmarshalStringNT(Marshaller.java:2348)
        at com.sas.iom.orb.brg.Marshaller.unmarshalParameter(Marshaller.java:1769)
        at com.sas.iom.orb.brg.Engine.insertArgs(Engine.java:1509)
        at com.sas.iom.orb.brg.Engine.insertRequestArgs(Engine.java:1468)
        at com.sas.iom.orb.brg.Engine.sendRequest(Engine.java:881)
        at com.sas.iom.orb.brg.RequestImpl.invoke(RequestImpl.java:37)
        at com.sas.meta.SASOMI._portable_stub_IOMI.GetMetadataObjects(_portable_stub_IOMI.java:94)
        at com.sas.metadata.remote.MdOMIWrapperImpl.getMetadataObjectsFromServer(MdOMIWrapperImpl.java:1731)
        at com.sas.metadata.remote.MdOMIWrapperImpl.getMetadataObjectsSubset(MdOMIWrapperImpl.java:685)
        at com.sas.metadata.remote.MdOMIUtilImpl.getMetadataObjectsSubset(MdOMIUtilImpl.java:1810)
        at com.sas.metadata.remote.MdOMIUtilImpl.getMetadataObjectsSubset(MdOMIUtilImpl.java:1792)
        at com.sas.workspace.mgmt.visuals.DBMSLibnamePropertyView.moveDataToView(DBMSLibnamePropertyView.java:239)
        at com.sas.workspace.WATabbedPanel.moveTabDataToView(WATabbedPanel.java:506)
        at com.sas.workspace.WATabbedPanel$cMoveDataToViewRunnable.run(WATabbedPanel.java:836)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733)
        at java.awt.EventQueue.access$200(EventQueue.java:103)
        at java.awt.EventQueue$3.run(EventQueue.java:694)
        at java.awt.EventQueue$3.run(EventQueue.java:692)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:703)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:154)
        at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:182)
        at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:221)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:219)
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:57)
        at java.nio.CharBuffer.allocate(CharBuffer.java:331)
        at com.sas.iom.orb.brg.Marshaller.decodeString(Marshaller.java:3261)
        at com.sas.iom.orb.brg.Marshaller.unmarshalStringNT(Marshaller.java:2348)
        at com.sas.iom.orb.brg.Marshaller.unmarshalParameter(Marshaller.java:1769)
        at com.sas.iom.orb.brg.Engine.insertArgs(Engine.java:1509)
        at com.sas.iom.orb.brg.Engine.insertRequestArgs(Engine.java:1468)
        at com.sas.iom.orb.brg.Engine.sendRequest(Engine.java:881)
        at com.sas.iom.orb.brg.RequestImpl.invoke(RequestImpl.java:37)
        at com.sas.meta.SASOMI._portable_stub_IOMI.GetMetadataObjects(_portable_stub_IOMI.java:94)
        at com.sas.metadata.remote.MdOMIWrapperImpl.getMetadataObjectsFromServer(MdOMIWrapperImpl.java:1731)
        at com.sas.metadata.remote.MdOMIWrapperImpl.getMetadataObjectsSubset(MdOMIWrapperImpl.java:685)
        at com.sas.metadata.remote.MdOMIUtilImpl.getMetadataObjectsSubset(MdOMIUtilImpl.java:1810)
        at com.sas.metadata.remote.MdOMIUtilImpl.getMetadataObjectsSubset(MdOMIUtilImpl.java:1792)
        at com.sas.workspace.mgmt.visuals.DBMSLibnamePropertyView.moveDataToView(DBMSLibnamePropertyView.java:239)
        at com.sas.workspace.WATabbedPanel.moveTabDataToView(WATabbedPanel.java:506)
        at com.sas.workspace.WATabbedPanel$cMoveDataToViewRunnable.run(WATabbedPanel.java:836)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733)
        at java.awt.EventQueue.access$200(EventQueue.java:103)
        at java.awt.EventQueue$3.run(EventQueue.java:694)
        at java.awt.EventQueue$3.run(EventQueue.java:692)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:703)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:154)
        at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:182)
        at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:221)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:219)
2016-01-22 10:27:47,206 [AWT-EventQueue-0] WARN  com.sas.workspace -
com.sas.services.session.LockingException: Session context is locked: [com.sas.services.plugins.publish.PublicationFrameworkPlugin].
        at com.sas.services.session.SessionContext.destroy(SessionContext.java:1130)
        at com.sas.workspace.connection.ConnectionManager.disconnectFromOMRServer(ConnectionManager.java:878)
        at com.sas.workspace.connection.ConnectionManager.disconnectFromOMRServer(ConnectionManager.java:847)
        at com.sas.workspace.Workspace.disconnectFromOMRServer(Workspace.java:3344)
        at com.sas.console.visuals.MainConsole.disconnectFromOMRServer(MainConsole.java:4662)
        at com.sas.workspace.Workspace.onWindowClosing(Workspace.java:5215)
        at com.sas.workspace.Workspace$1.windowClosing(Workspace.java:507)
        at java.awt.AWTEventMulticaster.windowClosing(AWTEventMulticaster.java:349)
        at java.awt.AWTEventMulticaster.windowClosing(AWTEventMulticaster.java:349)
        at java.awt.AWTEventMulticaster.windowClosing(AWTEventMulticaster.java:349)
        at java.awt.Window.processWindowEvent(Window.java:2051)
        at javax.swing.JFrame.processWindowEvent(JFrame.java:296)
        at java.awt.Window.processEvent(Window.java:2009)
        at java.awt.Component.dispatchEventImpl(Component.java:4861)
        at java.awt.Container.dispatchEventImpl(Container.java:2287)
        at java.awt.Window.dispatchEventImpl(Window.java:2719)
        at java.awt.Component.dispatchEvent(Component.java:4687)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:735)
        at java.awt.EventQueue.access$200(EventQueue.java:103)
        at java.awt.EventQueue$3.run(EventQueue.java:694)
        at java.awt.EventQueue$3.run(EventQueue.java:692)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
        at java.awt.EventQueue$4.run(EventQueue.java:708)
        at java.awt.EventQueue$4.run(EventQueue.java:706)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:705)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

[JuanS_OCS]

I have been expecting as this.

You are getting an error about excepting the jvm memory.

Maybe extending the jvm memory settings would be enough:http://support.sas.com/kb/12/864.html

Besides, it seems that the dataset is getting is too big: how big is your metadata repository? Did you analyzed and repaired your metadata? Are you running the metadata backup on reorganize mode?

[Prab]

HI,

Thanks for your comments.

We have below memory seetings in SMC.ini
JavaArgs_1=-Xmx1024m
JavaArgs_2=-XX:MaxPermSize=128mYes, we do have Backup and reorg activities.

PFA for groing size of Medatata size.

[JuanS_OCS]

Have you already tried to just double those values for the memory and see if there is any difference?

Your metadata is not very big. Anyway, I would strongly recommend to analyze your metadata repository.

If nothing if this is working, there is something corrupted in your metadata, we would need to come back to workarounds already proposed to you in this thread, much more "aggressive".

[Prab]

Sure ..Thank you very much for your answer.

We will check in that direction.

[Prab]

Sorry Attachment is here.