Problem in installing InspectIT about inspectit HOT 27 CLOSED

Hi,

nested exception is java.net.BindException: Address already in use: JVM_Bind

So you are already running something on the port.8182. Could you please check if the application that you are running on this port is necessary or could be shut down (are you running another inspectIT server instance?). Else you can reconfigure the server to run on another port that is free on your environment.

hth,
Stefan

from inspectit.

Hi,
Thanks for your answer! I've changed the port and it can start now. But I got another problem about the agent.
I can start the CMR and connect it with the UI. I've also configured the agent, but it seems that the agent cannot connect to the CMR.
The option I've added on the setEnv file of my WebLogic server is:
@set INSPECTIT_AGENT_HOME=D:\InspectIT\InspectIT\agent
@set EXTRA_JAVA_OPTIONS=-Xbootclasspath/p:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -javaagent:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -Dinspectit.repository=172.16.55.40:9071 -Dinspectit.agent.name=wlsagent
@set JAVA_OPTIONS=%JAVA_OPTIONS% %EXTRA_JAVA_OPTIONS%

In the system log, I found:
Jul 07, 2016 2:05:52 PM info.novatec.inspectit.agent.javaagent.JavaAgent premain
INFO: inspectIT Agent: Starting initialization...
Jul 07, 2016 2:05:54 PM info.novatec.inspectit.agent.config.impl.FileConfigurationReader load
INFO: Agent Configuration file not found at null\inspectit-agent.cfg, aborting!
Jul 07, 2016 2:05:54 PM info.novatec.inspectit.agent.PicoAgent init
SEVERE: The parser produced an exception!
info.novatec.inspectit.agent.config.ParserException: Agent Configuration file not found at null\inspectit-agent.cfg
at info.novatec.inspectit.agent.config.impl.FileConfigurationReader.load(FileConfigurationReader.java:141)
at info.novatec.inspectit.agent.PicoAgent.init(PicoAgent.java:123)
at info.novatec.inspectit.agent.PicoAgent.(PicoAgent.java:95)

Do you know where I did wrong? Can you tell me the possible measures I can take to make it works? Thanks in advance for your help.

from inspectit.

HI @xiaomutoun,

You are using the version of the inspectIT where configuration was specified in the configuration file (versions up to 1.6.7). However, the startup parameters for the agent:

@set INSPECTIT_AGENT_HOME=D:\InspectIT\InspectIT\agent
@set EXTRA_JAVA_OPTIONS=-Xbootclasspath/p:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -javaagent:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -Dinspectit.repository=172.16.55.40:9071 -Dinspectit.agent.name=wlsagent 
@set JAVA_OPTIONS=%JAVA_OPTIONS% %EXTRA_JAVA_OPTIONS%

are looking like the ones for the version 1.6.8+ where the agent configuration is moved to the user interface. So either update inspectIT to 1.6.8 (preferred) or add the

-Dinspectit.config=[PATH_TO_AGENT]/config

to the list of start-up parameters.

from inspectit.

Hi @ivansenic,

Thanks for your answer, I've changed to version 1.6.8 and I've succeed in installing InspectIT.
But I encounter another problem. In fact, I instrumented the agent on one of the frontends of my server by adding the Java options. After the instrumentation, InspectIT can detect the agent and monitor some system resources, but the server can't function well. There comes some new exceptions like:
<An exception occurred while creating the free pool for EJB airtransfer(Application: frontendapp, EJBComponent: etv-ejb/airtransfer.jar). Since this occurred during server start-up, when a
dependent resource may be unavailable, the error will be ignored and deployment will proceed. The EJB container will instead attempt to create the bean instances on demand.

and database connection pool exception:
com._.configuration.ConfigurationException: The provided path is not valid: com._.util.databasehandler.ConnectionPooling

The problem is that these exceptions do not happen without the instrumentation of InspectIT. So do you know the possible reason of that? Do you know what possible measures could I take to deal with it? Thanks in advance for your help.

from inspectit.

Hi @xiaomutoun,

Sure we would like to help. Can you for start send us the complete stack trace of exceptions you get? This would help a lot because otherwise I can not really know what's going on. Also can you send me any details of how are you running your app (application server, standalone java, etc)?

from inspectit.

Hi @ivansenic,

It would be very kind if you could help me to analyse the log.
In my case, I have four machines: a web server, two WebLogic server with 2 backends and 2 frontends on each, and a database machine. I added the option to setEnv:
@set INSPECTIT_AGENT_HOME=D:\InspectIT\InspectIT2\agent
@set EXTRA_JAVA_OPTIONS=-Xbootclasspath/p:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -javaagent:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -Dinspectit.repository=172.16.55.40:9070 -Dinspectit.agent.name=wlsagent

And InspectIT can detect the agent and monitor some system resources.
Here are the logs for without InspectIT and with InspectIT.
Thanks a lot for your help.

from inspectit.

Hi @xiaomutoun,

It seams that the setup is correct and agents starts correctly. Can you also send me the logs that are created in the [AGENT_BASE]/logs folder. This would also be helpful. As soon I get the idea what's going on wrong I'll let you know.

from inspectit.

Hi @ivansenic,

Here is the log in agent base.

from inspectit.

Hi @xiaomutoun,

Hmm although there are no error logs related to inspectIT in the files you gave me, I have a suspicion what can be causing this problem. Due to the bug (https://inspectit-performance.atlassian.net/browse/INSPECTIT-2119) in version 1.6.8 it might be that there is additional setting needed to make agent work.

That's related to the OSGi. Do you start the OSGi container with your Weblogic? If so can you add the rocks.inspectit to the org.osgi.framework.bootdelegation property? Here s page on how to configure that: https://docs.oracle.com/middleware/1212/wls/WLPRG/osgi.htm#WLPRG794

It would be great if you can test it and let us know if it worked. Please note that on Friday we are releasing the new version that will have this bug fixed and there would be no need for such a hack :)

from inspectit.

Hi @ivansenic,

Thanks. I will try it and give you the result.

from inspectit.

Hi @ivansenic,

I've downloaded the new version, 1.6.9.83. And it seems that there is still some problem. Here is the log, it would be great if you could help me figure out what causes it. Thanks in advances.

from inspectit.

Hi @xiaomutoun,
please remove the -Xbootclasspath/p: ... option from the EXTRA_JAVA_OPTIONS (but leave the -javaagent:...) and try it again. As you use spring, the problems could be related to this.

from inspectit.

Hi @pbouillet,
Thanks for your help. I've removed the option, but it seems that there is no change ...

from inspectit.

Thanks for trying out! I really wonder why this could even happen. Did you increase the log information betwenn the run without inspectit and with? Because the log with inspectit is way more detailed. If so, could you start again just without the javaagent parameter so that comparing is easier? Thanks in advance!

from inspectit.

Hi,
This is the log that I removed the option of InspectIT. There is some exception, but not as much as that with InspectIT. It's very kind of you to help me to analyze it.

from inspectit.

This is very interesting that the log output differs that much. I currently don't have any idea to try out something. I need to discuss this with Ivan on monday. I will let you know what to do next.

from inspectit.

Hi @xiaomutoun,

Thanks a lot for all the submissions, it's really important for us to solve this, as we don't get quite often our hands on Weblogic. Thus, we'll try our best to help.

I created an agent with small updates inspectit-agent-sun1.5-1.6.0.zip. In this agent I disabled completely JMX sensor (as we already have some bugs related to it) and I increased logging information for the class loaders that are used when our agent is initialized. We just want to make sure that everything there is OK, as we saw that somehow we interfere with your sysout log file. This agent is based on the version 1.6.9 (ignore the 1.6.0.0 thing) so you don't have to change CMR/UI, just use the ones you have. Try it out and send us the logs please (server, sysout and ones in the agent/logs folder).

The second thing that we would like you to try is to change a bit the instrumentation configuration. In the UI please go to the Configuration perspective and open the Default Environment. There disable the [Common] SQL and [Common] HTTP profiles and also un-check the options Class loading delegation (located under Strategies section). This would remove all instrumentations that might be there, so try again and check if this solves it. If it does maybe try to activate/deactivate around these 3 options and figure out what can be the one causing the problem. If we figure out this we would be half way there I would say.

Once again thanks for submitting this and we hope we can solve it soon 😄

from inspectit.

Hi @ivansenic,
Here are the new logs. I use your new agent, and make two test, one with the three options, one without them. But I think there isn't so much difference. Could you please help me analyze them? Thanks.

from inspectit.

Hi @xiaomutoun,

This is very strange, seams like the problems are still there although we removed all instrumentations. We still have a theory that something is wrong with the log initialization and that it breaks the server. We tried it this morning on the clean WLS 10.3.6 and we also hit one exception (which you don't) on the log initialization. So I built up new agent with no log initialization at all: inspectit-agent-sun1.5-1.6.0.zip. Since there is no setup this will print everything that is going on in inspectIT, so be prepared for lot of statements. Anyhow with this new agent it works for us on the clean WSL, we get all the data and server works normally.

If this does not fix the problem, then it must be some specifics in your application or configuration. Can you tell us any property you might configured related to class loaders, security policies, etc. Everything that comes to you mind?

Thanks for helping,
Ivan

from inspectit.

Hi,
Thanks for your help. I've used the new agent that you provide, but it seems that the problem remains.
Here is the log with a new file sendingClass.cache created by agent.
I'll look for the property about class loaders etc, and I will give you the information once I get it.

from inspectit.

Hi,

small question, which logging framework do you use to print these statements:

12:27:06,934 [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO   CdpStartup : loading all the startup classes

And the server and applications are still not functioning correctly, right? (I know that the exceptions in the log are still there, just want to identify if they really affect the server or are there maybe anyway which you normally would not see at all).

from inspectit.

Hi @pbouillet,
I am sorry that I don't know much deep about the server. So could you please tell me how I can see which logging framework I am using?

from inspectit.

Hi all,
I asked one of my collegues, and changed the jvm option. It seems to improve something.
Before, the option is:
@set EXTRA_JAVA_OPTIONS= -javaagent:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -Dinspectit.repository=172.16.55.40:9070 -Dinspectit.agent.name=wlsagent
and it should be:
@set EXTRA_JAVA_OPTIONS= %EXTRA_JAVA_OPTIONS% -javaagent:%INSPECTIT_AGENT_HOME%\inspectit-agent.jar -Dinspectit.repository=172.16.55.40:9070 -Dinspectit.agent.name=wlsagent
After that, there are still some exceptions, but I can run the Vuser scripts and perform some tests.
Thanks all for your help.

from inspectit.

Hi @xiaomutoun,

That's great that it works now.. We obviously missed the overwriting of the EXTRA JAVA OPTIONS.. Can you just tell us if it also works with the non-modified agent in the version 1.6.9 or you are using one of the agents we provided?

Thanks again for all the help.. I hope inspectIT will help you in your tests.. If you need any more help let us know..

Cheers

from inspectit.

Hi @ivansenic,
I am now using the original agent in your new release of last week, with all the three instrumentations options on. It seems to work now. Thanks again for your help.

from inspectit.

Hi , I am trying to install the installIT in local machine and integrate with my Project.
I run the inspectit.installer-all.macosx.x64-1.7.7.90.jar. and after that went to CMR folder and started the CMR.

I am facing two problem

1. I am not sure where to get the UI release and install it.
2. How to connect the CMR with UI.

Pls help me on this

from inspectit.

Hi @v71017 , did you take a look at our documentation at https://inspectit-performance.atlassian.net/wiki/display/DOC17/Installation+Guide ? Besides the CMR folder, there is a folder inspectIT in which the UI is located.

If you have further questions, come to our Gitter chat at https://gitter.im/inspectIT/chat or create a new issue. This one has been closed some time ago already, thanks!

from inspectit.