Java Trace file analytics - Newbie

Hello,
I am new to the exciting AI/ML world and recently got exposed to Knime. I have a pretty steep learning curve ahead of me but wanted to get some head start from all you experts.
We have a Java application that spits out continuous logs into a trace file, and was exploring if we can do any predictive analytics and anomaly detection based on it. The challenge is data preparation, since the trace files has whole bunch of unwanted information. I want date, time, some specific events and user id from the trace files and try to use it.
I am sure this is nothing new, but if you can guide me how to approach and tackle would be greatly appreciated. Below is the excerpt of the trace file and some information in bold which i would to extract and do some kind of analytics on them. I have many of them generated historically,:

#2.0#2023 02 06 03:28:51:593#+00#Error#com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl#
com.sap.ASJ.jmx_srv.000011#BC-JAS-ADM-ADM#jmx#C000AC182898FE190000000100003ED5#9499650000000002##com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl#SM_COLL_PS1#293182##45C2B8BACD1810069BCEDE000C63DB8A#45c2b8bacd1810069bcede000c63db8a##0#Thread[RMI/IIOP Worker [3],5,Dedicated_Application_Thread]#Plain##
Runtime exception occurred while processing external JMX request [ JMX request (java) v1.0 len: 470 | src: 2 req: getAttribute params-number: 2 params-bytes: 0 | com.sap.default:version=1.0,SAP_ITSAMGalaxyTaskStatisticsProviderMBean.Name=SAP_ITSAMGalaxyTaskStatisticsProviderMBean_PX1.SystemHome.zts-piopl01,type=SAP_ITSAMJ2eeCluster.SAP_ITSAMGalaxyTaskStatisticsProviderMBean,cimclass=SAP_ITSAMGalaxyTaskStatisticsProviderMBean,SAP_ITSAMJ2eeCluster.Name=PX1.SystemHome.zts-piopl01,SAP_ITSAMJ2eeCluster.CreationClassName=SAP_ITSAMJ2eeCluster Volumes ]
com.sap.engine.services.jmx.exception.JmxSecurityException: Caller SM_COLL_PS1 not authorized, required permission missing (“javax.management.MBeanPermission” “-#Volumes[com.sap.default:SAP_ITSAMGalaxyTaskStatisticsProviderMBean.Name=SAP_ITSAMGalaxyTaskStatisticsProviderMBean_PX1.SystemHome.zts-piopl01,SAP_ITSAMJ2eeCluster.CreationClassName=SAP_ITSAMJ2eeCluster,SAP_ITSAMJ2eeCluster.Name=PX1.SystemHome.zts-piopl01,cimclass=SAP_ITSAMGalaxyTaskStatisticsProviderMBean,type=SAP_ITSAMJ2eeCluster.SAP_ITSAMGalaxyTaskStatisticsProviderMBean,version=1.0]” “getAttribute”)
at com.sap.engine.services.jmx.auth.UmeAuthorization.checkMBeanPermission(UmeAuthorization.java:106)
at com.sap.engine.services.jmx.JmxServerFrame.checkMBeanPermission(JmxServerFrame.java:101)
at com.sap.engine.services.jmx.MBeanServerSecurityWrapper.checkMBeanPermission(MBeanServerSecurityWrapper.java:438)
at com.sap.engine.services.jmx.MBeanServerSecurityWrapper.getAttribute(MBeanServerSecurityWrapper.java:234)
at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:136)
at com.sap.engine.services.jmx.ClusterInterceptor.invokeMbs(ClusterInterceptor.java:220)
at com.sap.engine.services.jmx.ClusterInterceptor.getAttribute(ClusterInterceptor.java:550)
at com.sap.engine.services.jmx.MBeanServerInterceptorInvoker.invokeMbs(MBeanServerInterceptorInvoker.java:85)
at com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl.invokeMbs(P4ConnectorServerImpl.java:61)
at com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImplp4_Skel.dispatch(P4ConnectorServerImplp4_Skel.java:64)
at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:483)
at com.sap.engine.services.rmi_p4.server.ServerDispatchImpl.run(ServerDispatchImpl.java:83)
at com.sap.engine.services.rmi_p4.P4Message.process(P4Message.java:72)
at com.sap.engine.services.rmi_p4.P4Message.execute(P4Message.java:43)
at com.sap.engine.services.cross.fca.FCAConnectorImpl.executeRequest(FCAConnectorImpl.java:1055)
at com.sap.engine.services.rmi_p4.P4Message.process(P4Message.java:59)
at com.sap.engine.services.cross.fca.MessageReader.run(MessageReader.java:55)
at com.sap.engine.core.thread.execution.Executable.run(Executable.java:122)
at com.sap.engine.core.thread.execution.Executable.run(Executable.java:101)
at com.sap.engine.core.thread.execution.CentralExecutor$SingleThread.run(CentralExecutor.java:328)

[Log ID = C000AC182898FE190000000000003ED5]#

#2.0#2023 02 06 03:28:55:567#+00#Error#com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl#
com.sap.ASJ.jmx_srv.000011#BC-JAS-ADM-ADM#jmx#C000AC182898FE200000000100003ED5#9499650000000002##com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl#SM_COLL_PS1#293185##32714696CD191006A74EB80BAA95F682#32714696cd191006a74eb80baa95f682##0#Thread[RMI/IIOP Worker [4],5,Dedicated_Application_Thread]#Plain##
Runtime exception occurred while processing external JMX request [ JMX request (java) v1.0 len: 684 | src: 2 target-node: 9499650 req: getAttribute params-number: 2 params-bytes: 0 | com.sap.default:SAP_ITSAMJ2eeNode.Name=9499650,SAP_ITSAMJ2eeCluster.CreationClassName=SAP_ITSAMJ2eeCluster,SAP_J2EEClusterNode=9499650,SAP_ITSAMJ2eeInstance.CreationClassName=SAP_ITSAMJ2eeInstance,SAP_ITSAMJ2eeNode.CreationClassName=SAP_ITSAMJ2eeNode,SAP_ITSAMJ2eeInstance.Name=PX1.InstanceID.00.SystemHome.zts-piopl01,SAP_ITSAMJ2eeCluster.Name=PX1.SystemHome.zts-piopl01,version=1.0,type=SAP_ITSAMJ2eeCluster.SAP_ITSAMJ2eeInstance.SAP_ITSAMJ2eeNode.SAP_ITSAMGalaxyKernelMonitor,SAP_ITSAMGalaxyKernelMonitor.Name=SAP_ITSAMGalaxyKernelMonitor_9499650,cimclass=SAP_ITSAMGalaxyKernelMonitor TrafficLights ]
com.sap.engine.services.jmx.exception.JmxSecurityException: Caller SM_COLL_PS1 not authorized, required permission missing (“javax.management.MBeanPermission” “-#TrafficLights[com.sap.default:SAP_ITSAMGalaxyKernelMonitor.Name=SAP_ITSAMGalaxyKernelMonitor_9499650,SAP_ITSAMJ2eeCluster.CreationClassName=SAP_ITSAMJ2eeCluster,SAP_ITSAMJ2eeCluster.Name=PX1.SystemHome.zts-piopl01,SAP_ITSAMJ2eeInstance.CreationClassName=SAP_ITSAMJ2eeInstance,SAP_ITSAMJ2eeInstance.Name=PX1.InstanceID.00.SystemHome.zts-piopl01,SAP_ITSAMJ2eeNode.CreationClassName=SAP_ITSAMJ2eeNode,SAP_ITSAMJ2eeNode.Name=9499650,SAP_J2EEClusterNode=9499650,cimclass=SAP_ITSAMGalaxyKernelMonitor,type=SAP_ITSAMJ2eeCluster.SAP_ITSAMJ2eeInstance.SAP_ITSAMJ2eeNode.SAP_ITSAMGalaxyKernelMonitor,version=1.0]” “getAttribute”)
at com.sap.engine.services.jmx.auth.UmeAuthorization.checkMBeanPermission(UmeAuthorization.java:106)
at com.sap.engine.services.jmx.JmxServerFrame.checkMBeanPermission(JmxServerFrame.java:101)
at com.sap.engine.services.jmx.MBeanServerSecurityWrapper.checkMBeanPermission(MBeanServerSecurityWrapper.java:438)
at com.sap.engine.services.jmx.MBeanServerSecurityWrapper.getAttribute(MBeanServerSecurityWrapper.java:234)
at com.sap.engine.services.jmx.MBeanServerInvoker.invokeMbs(MBeanServerInvoker.java:136)
at com.sap.engine.services.jmx.ClusterInterceptor.invokeMbs(ClusterInterceptor.java:220)
at com.sap.engine.services.jmx.ClusterInterceptor.getAttribute(ClusterInterceptor.java:550)
at com.sap.engine.services.jmx.MBeanServerInterceptorInvoker.invokeMbs(MBeanServerInterceptorInvoker.java:85)
at com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImpl.invokeMbs(P4ConnectorServerImpl.java:61)
at com.sap.engine.services.jmx.connector.p4.P4ConnectorServerImplp4_Skel.dispatch(P4ConnectorServerImplp4_Skel.java:64)
at com.sap.engine.services.rmi_p4.DispatchImpl._runInternal(DispatchImpl.java:483)
at com.sap.engine.services.rmi_p4.server.ServerDispatchImpl.run(ServerDispatchImpl.java:83)
at com.sap.engine.services.rmi_p4.P4Message.process(P4Message.java:72)
at com.sap.engine.services.rmi_p4.P4Message.execute(P4Message.java:43)
at com.sap.engine.services.cross.fca.FCAConnectorImpl.executeRequest(FCAConnectorImpl.java:1055)
at com.sap.engine.services.rmi_p4.P4Message.process(P4Message.java:59)
at com.sap.engine.services.cross.fca.MessageReader.run(MessageReader.java:55)
at com.sap.engine.core.thread.execution.Executable.run(Executable.java:122)
at com.sap.engine.core.thread.execution.Executable.run(Executable.java:101)
at com.sap.engine.core.thread.execution.CentralExecutor$SingleThread.run(CentralExecutor.java:328)

Wanted to give few more information -
All new logs are separated by a log id - e.g. [Log ID = C000AC182898FE550000000000003ED5]#
The trace files start like this - !–LOGHEADER[START]/–>

Hello Shamiknit,

In order to extract precise information from a log file, we may require additional details. For example, we would need to understand the text pattern in the log file and how the desired data is distributed within it.

You can refer to this link as an example of how to extract data from a log file. I hope this is helpful.

Best regards, Jinwei

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.