What is the best APM (Application Performance Management) solution out there in the market today? Unfortunately that question cannot be answered until your requirements are clearly defined. However, when shopping for an APM solution, there are certain qualities that you must consider in order to ensure maximum ROI (Return on Investment).

The following are 8 qualities you must look for in an APM solution if your enterprise is primarily Java based:

1. Low Overhead

Almost all APM solutions for the Java platform are agent based. This means a piece of software (a jar file) is installed in your application which will collect metrics and send them to a server application where metrics are processed. It is critical to make sure the agent does not exert too much load on the application. Based on my experience, CPU utilization overhead of 3% is acceptable.

The fact of the matter is, even though the "out of the box" agent overhead can be very small, one can easily misconfigure the agent (for example, using an incorrect JMX filter to collect JMX metrics) that results in increased overhead. So, pay attention to the agent configuration that defines the instrumentation.

2. JVM Memory monitoring and diagnostics

JVM Heap is perhaps the most important metric to be monitored as far as JVM health is concerned. When the JVM runs out of the heap memory, "OutOfMemory" error is thrown which in most of the cases, leads to degraded performance or complete hang up. It is not just enough to monitor the size of the Heap, the APM solution must monitor at least the following metrics related to JVM Memory monitoring:

a. Heap usage

b. Garbage collection duration (minor and major collections)

c. Garbage collection frequency (minor and major collections)

d. Various memory pools usage:
i. Eden
ii. Survivor
iii. Old gen
iv. Permgen
v. Code cache

A nice-to-have diagnostic feature is to identify memory leak suspects. Basically, a collection object being alive for long time or a collection object of unusually large size would indicate a possible memory leak. If the APM tool automatically recognizes a leaking object, it can be of great use to prevent "OutOfMemory" errors.

Note: With Java 8, permgen has been replaced by metaspace.

3. Backend response times

A typical Java enterprise web application uses standard APIs to access various backend resources. For example:

1. A RDBMS such as Oracle or MS SQL Server using JDBC, Hibernate and JCA API

2. A JMS Server such as MQ Server using JMS and JCA API

3. A Web Services Server using SOAP, Web Services API

4. An ERP Server such as SAP using JCA and vendor provided drivers

The APM tool must automatically recognize these backend calls and monitor the response times.

For RDBMS, the APM tool must capture the actual SQL statements and names of the stored procedures.

4. Ease of Deployment

The installation of the agent should not be time consuming and should be scriptable. It is one thing to install 5 agents and entirely another thing to install hundreds of agents.

Here are the questions to ask when it comes to deploying the agents:

a. How many steps are involved?

b. Can the process be scripted?

c. Can the agents be automatically named so that they are sure to be unique?

d. Does the process require multiple restarts of the application server?

5. Transaction monitoring across JVMs

In today’s enterprise, it is not unusual to have a system of several application servers performing different business functions. Therefore, the APM tool must be able to correlate and track a transaction across several JVMs. This capability is important because this forms the entire transaction execution path and thus reveals the weakest link.

6. Code level visibility

The APM tool must be able to locate the line of code that is causing the issue. This is where we get the real bang for the buck. For example, simply stating "order processing is slow" does not help a lot. However, if the APM tool can reveal the point at which the transaction is taking the most amount of time (for example: class: com.mycompany.orders.preprocessOrder.java method: checkAvailability Line 172), it will be much easier for the developer to know where to look.

7. Dynamic instrumentation

If you need to make a change to your instrumentation, for example, if you need to add a new method to be monitored, does the JVM need to be restarted? Starting from JRE 1.5, Java supports dynamic instrumentation. Make sure your APM tool can dynamically update the instrumentation. If you need to restart the JVM every time you change the configuration, it will be extremely difficult to manage.

8. World class support

With fierce competition in the APM market, if a feature is introduced by one vendor, it is immediately made available by other vendors as well. What makes the APM vendors different is the support they provide. Make sure the support is world class with quick turnaround times. You really want a vendor that listens to your needs and works diligently to fulfill them.

There you have it. The 8 requirements to look for in an APM solution, if yours is primarily an Enterprise Java shop. You will get the best ROI when all 8 requirements are met by an APM solution. Work with the vendor during your POC (proof of concept) process to thoroughly vet the 8 requirements. With world class support from the vendor (which is one of the requirements mentioned), you will be up and running in no time.

ABOUT Karun Subramanian

Karun Subramanian is the owner and CEO of ESIS Consulting LLC. He is an application support expert with 16 years of experience, and a major focus on APM technologies. He is a certified CA Wily Introscope Analyst and has helped several fortune 100 companies solve their performance problems using a variety of tools such as AppDynamics, Solarwinds, Nagios, IBM ITCAM, HP Open View etc.