GC Log analysis compliments APM
Recently a smart engineering manager from a major financial institution asked the question: ‘We are already using APM tool (AppDynamics), why do we need to use GC log analysis tool (GCeasy)?’. Fair question that deserves detailed answer. There are angles to this question:
(a). Purpose
(b). Metrics
Let’s review them in this article.
(a). Purpose
APM tools are fantastic tools to monitor your applications performance, health in production environment. They give entire big picture on application’s availability, performance in production environment. However, they aren’t meant to optimize & tune your application’s Garbage Collection and memory settings.
APM tools are typically meant to be used in production environment. On the other hand, GC log analysis tools can be used in Dev, Test and production environment.
(b). Metrics
GC Log analysis tools by provides rich set of Garbage collection, memory related micrometrics (which aren’t provided by APM tools). Here is a overview of such metrics that are provided only by GCeasy.
1. Garbage Collection KPIs
There are 3 Key Performance Indicators when it comes to Garbage Collection analysis:
- Throughput
- Latency (i.e. GC Pause time)
- Footprint
APM tools reports only Footprint, but not other two critical KPI. More details on each Garbage Collection KPI is described in this article.
Fig: GC KPIs reported by GCeasy
2. Phase metrics
Within one single GC event there are multiple phases. Example G1 GC algorithm has following phases: initial mark, root region scanning, concurrent marking, remark, cleanup. CMS GC algorithm has following phases: initial mark, concurrent mark, concurrent preclean, concurrent abortable preclean, final remark, concurrent sweep, concurrent reset.
APM tools doesn’t give metrics on these individual phases, whereas GCeasy provides detailed metrics on each phase. These metrics are crucial when you are tuning, optimizing the GC/memory settings of your application.
Fig: Individual GC Phases metrics generated by GCeasy
3. GC Causes
Garbage collection by itself doesn’t add value in processing customer transactions. It is a necessary evil, to clean-up unreferenced objects, make room to handle new incoming requests. However, Garbage collection consumes considerable amount of CPU and causes application to pause. Thus, one should try to reduce number of time GC events run and amount of time it takes to run.
GCeasy reports the reasons that are triggering Garbage collection events. If these reasons can be addressed GC event counts and pause times, caused by them can be minimized. These ‘GC Causes’ aren’t reported by APM tools.
Fig: Reasons triggering GC events
4. ML algorithms to detect problems
GCeasy employs machine learning algorithms to detect various memory/GC related problems. Detected problems are reported in report. GCeasy not only detects problems, but it also recommends solutions to fix the detected problems. Below are few such problems and solutions reported by GCeasy tool:
Fig: GCeasy detects GC/memory related problems automagically
5. ML algorithms – memory tuning recommendations
GCeasy’s machine learning algorithms provides tips for optimizing and tuning GC and memory related settings in the JVM.
Fig: GC Tuning tips recommended by GCeasy
6. REST API – for CI/CD Integration
GCeasy provides JSON based REST API to analyze Garbage collection logs. Instead of manually uploading GC logs and analyzing, you can use this API to analyze hundreds/thousands of GC logs in automated manner. API provides detailed micrometrics on various aspects of application’s memory and GC utilization. These APIs are used by enterprises in their CI/CD pipeline. It facilitates to catch various GC and memory related performance problems right during code commit time.
7. Amount of memory wasted
Today enterprises wastes anywhere between 30 – 80% of memory because of inefficient programming practices such as: Duplication of String, Inefficient Data structure implementation, Suboptimal data type definitions, overallocation and underutilization of generation sizes…. GCeasy’s sister product HeapHero is the first tool in the industry to report the amount of memory wasted due to inefficient programming.
Fig: HeapHero reporting amount of memory wasted due to inefficient programming Practices.
Conclusion
Garbage Collection log analysis tools such as GCeasy, Garbage Cat, HP Jmeter aren’t replacement to APM tools such as AppDynamics, NewRelic, Dynatrace, Wily Introscope. Garbage collection log analysis tool compliments APM tools by providing additional metrics and purposes.
Author bio:
Ram Lakshmanan
Every single day, millions & millions of people in North America—bank, travel, and commerce—use the applications that Ram Lakshmanan has architected. Ram is an acclaimed speaker in major conferences on scalability, availability, and performance topics. Recently, he has founded a startup, which specializes in troubleshooting performance problems.
