Русские Блоги
Итак, обычно есть две причины этой аномалии:
1. В программе есть бесконечный цикл.
2. Программа занимает слишком много памяти, что превышает максимальное значение, установленное кучей JVM.
В первом случае вам необходимо самостоятельно проверить программный код, поэтому я не буду здесь говорить больше.
Во втором случае мы вручную расширяем настройки параметров кучи JVM. Настройка кучи JVM относится к настройке пространства памяти, которое JVM может выделить и использовать во время выполнения программы java. Когда JVM запускается, куча JVM автоматически устанавливает значение размера кучи. Обычно значение по умолчанию для начального пространства (например, -Xms) составляет 1/64 физической памяти, а максимальное пространство составляет 1/4 физической памяти. Его можно установить с помощью таких параметров, как -Xmn -Xms -Xmx, предоставляемых JVM. Вот объяснение значения каждого параметра:
-Xms: начальное значение
-Xmx: максимум
-Xmn: минимальное значение
Размер кучи не должен быть слишком маленьким или слишком большим. Если параметр слишком мал, скорость отклика программы будет ниже, потому что сборщик мусора занимает больше времени, а приложение выделяет меньше времени на выполнение. Слишком большой размер также приведет к потере места и повлияет на нормальную работу других программ. Размер кучи не должен превышать 80% доступной физической памяти. Рекомендуется установить одинаковые параметры -Xms и -Xmx, а -Xmn составляет 1/4 значения -Xmx.
Основные методы настройки следующие:
1. Этот параметр добавляется при выполнении файла класса JAVA, где className — это имя класса, который необходимо выполнить. (Включая имя пакета) Например: java -Xms32m -Xmx800m className Это не только решает проблему, но и скорость выполнения намного выше, чем когда она не установлена. Если это тест разработки, вы также можете установить его прямо в eclipse. Введите -Xms32m -Xmx800m в аргументы виртуальной машины в Eclipse -> run -arguments.
2. Вы можете изменить системные переменные среды в Windows и добавить JAVA_OPTS = -Xms64m -Xmx512m.
3. Если вы используете tomcat под окнами, вы можете добавить в C: \ tomcat5.5.9 \ bin \ catalina.bat (конкретный путь зависит от местоположения вашего tomcat): установить JAVA_OPTS = -Xms64m -Xmx256m (размер зависит от вашей собственной памяти) Местоположение: rem Угадайте CATALINA_HOME, если не определено Добавьте соответствующее в этой строке.
4. Если это система Linux, добавьте набор JAVA_OPTS = ’- Xms64 -Xmx512’ перед
Поскольку программе необходимо прочитать около 10 Вт строк записей из данных для обработки, возникает ошибка типа java.lang.OutOfMemoryError: пространство кучи Java появляется при чтении 9 Вт.
Проверка в Интернете может быть причиной того, что параметр стека JAVA слишком мал.
Согласно ответам в Интернете, существует примерно два решения:
1. Задайте переменные среды.
set JAVA_OPTS= -Xms32m -Xmx512m
можно изменить в соответствии с объемом памяти вашего компьютера, но моя проверка этого метода не решила проблему. Это может быть где еще нужно установить.
2、java -Xms32m -Xmx800m className
— добавить этот параметр при выполнении файла класса JAVA, где className — это фактическое имя класса, который должен быть выполнен. (Включая название пакета)
Это решает проблему. И скорость выполнения намного выше, чем без настройки.
Если вы можете использовать Eclispe при тестировании, вам необходимо ввести параметр -Xms32m -Xmx800m в аргументы виртуальной машины в Eclipse -> run -arguments.
java.lang.OutOfMemoryError: Java heap space
Исключение возникает при использовании программы Java для запроса большого количества данных из базы данных:
java.lang.OutOfMemoryError: Java heap space
В JVM, если 98% времени используется для сборки мусора, а доступный размер кучи меньше 2%, будет выдано это сообщение об исключении.
Настройка кучи JVM относится к настройке пространства памяти, которое JVM может выделить и использовать во время выполнения программы java.
JVM автоматически установит значение размера кучи при запуске.Его начальное пространство (-Xms) составляет 1/64 физической памяти, а максимальное пространство (-Xmx) — 1/4 физической памяти. Его можно установить с помощью таких параметров, как -Xmn -Xms -Xmx, предоставляемых JVM.
Например: java -jar -Xmn16m -Xms64m -Xmx128m MyApp.jar
Если размер кучи установлен слишком маленьким, в дополнение к этим аномальным сообщениям вы обнаружите, что скорость отклика программы снижается. Сборщик мусора занимает больше времени, а приложение выделяет меньше времени на выполнение.
Размер кучи не должен превышать 80% доступной физической памяти.Обычно для параметров -Xms и -Xmx должны быть установлены одинаковые значения, а -Xmn составляет 1/4 значения -Xmx.
Параметры размера кучи -Xms -Xmn не должны превышать размер физической памяти. В противном случае появится сообщение «Ошибка при инициализации виртуальной машины. Не удалось зарезервировать достаточно места для кучи объектов».
==========================================================
После ночи напряженной работы я наконец завершил программу замены файлов для указанной строки, но поскольку я хочу заменить слишком много файлов html для общесайтовой программы, поэтому затмение всегда заканчивается в каталоге После сообщения об исключении java.lang.OutOfMemoryError: пространство кучи Java произошел сбой.
Я подумал, что слишком поздно перерабатывать из-за частых операций, поэтому я добавил Thread.sleep (1000) после каждого цикла и обнаружил, что он умрет в этом каталоге, поэтому я изменил 1000 на 5000 или умер там. Я думаю, что это может быть не так просто перерабатывать, возможно, JVM Sun просто не выпускает для этой ситуации.
Затем я добавил -Xmx256M к параметру запуска, на этот раз все было нормально.
Подумав об этом, я до сих пор мало что знаю о принципах сборки мусора, я проверил это в Интернете и нашел несколько хороших статей.
Также существуют: Управление кучей Java — сборка мусора. Следует отметить следующие моменты, которые могут использоваться в качестве рекомендаций при написании программ:
(1) Не пытайтесь предполагать время, когда происходит сборка мусора, причем все это неизвестно. Например, временный объект в методе становится бесполезным после вызова метода, и его память может быть освобождена в это время.
(2) Java предоставляет несколько классов, которые занимаются сборкой мусора, и предоставляет способ принудительного вызова функции сборки мусора System.gc (), но это также ненадежный метод. Java не гарантирует, что сборка мусора будет запускаться каждый раз при вызове этого метода. Она просто отправляет такой запрос в JVM. Неизвестно, выполняется ли сборка мусора на самом деле.
(3) Выберите подходящий вам сборщик мусора. Вообще говоря, если система не предъявляет особых требований к производительности, вы можете использовать параметры JVM по умолчанию. В противном случае вы можете рассмотреть возможность использования целевых сборщиков мусора.Например, инкрементные сборщики больше подходят для систем с высокими требованиями к работе в реальном времени. Система имеет более высокую конфигурацию и больше простаивающих ресурсов, вы можете рассмотреть возможность использования параллельного сборщика меток / разверток.
(4) Ключевая и сложная проблема — это утечки памяти. Хорошие навыки программирования и строгое отношение к программированию всегда являются самыми важными. Не позволяйте небольшой собственной ошибке вызвать большую дыру в памяти.
(5) Освободите ссылки на бесполезные объекты как можно скорее.
Когда большинство программистов используют временные переменные, они автоматически устанавливают для ссылочной переменной значение null после выхода из активной области (области), что означает, что сборщик мусора будет собирать объект. Вы должны обратить внимание на то, отслеживается ли объект, на который указывает ссылка, если да, удалите прослушиватель, а затем назначьте нулевое значение.
Другими словами, лучше контролировать операции частого обращения к памяти и освобождения памяти самостоятельно, но метод System.gc () может быть неприменим. Лучше использовать finalize для принудительного выполнения или написать свой собственный метод finalize.
Я обнаружил ошибку TOMCAT: java.lang.OutOfMemoryError: пространство кучи Java, поэтому я проверил информацию и нашел решение:
If Java runs out of memory, the following error occurs:
Exception in thread “main” java.lang.OutOfMemoryError: Java heap space
Java heap size can be increased as follows:
java -Xms -Xmx
Defaults are:
java -Xms32m -Xmx128m
Если вы используете выигрыш
/tomcat/bin/catalina.bat плюс следующая команда:
set JAVA_OPTS=-Xms32m -Xmx256m
Если вы используете unix / linux
/tomcat/bin/catalina.sh плюс следующая команда:
JAVA_OPTS="-Xms32m -Xmx256m"
инструмент просмотра и анализа памяти jvm
В отрасли существует множество мощных инструментов для профилей Java, таких как Jporfiler и yourkit. Я не хочу говорить об этих платных вещах. Я хочу сказать, что сама java обеспечивает большой мониторинг памяти. Маленькие инструменты, перечисленные ниже инструменты — лишь небольшая часть. Все еще довольно интересно внимательно изучить инструменты jdk 🙂
1: вывод журнала gc
-verbose: gc и -XX: + PrintTenuringDistribution и т. д.
Код коллекции HTML-кода
Usage:
jmap -histo (to connect to running process and print histogram of java object heap
jmap -dump: (to connect to running process and dump java heap)
dump-options: format=b binary default file=
dump heap to
Example: jmap -dump:format=b,file=heap.bin
jmap -dump:file=c:\dump.txt 340
Обратите внимание, что 340 — это pid java-процесса моей машины. Размер выгруженного файла превышает 10 мегабайт, и я только что открыл tomcat и запустил очень простое приложение без какого-либо доступа. Его можно представить на большом и загруженном сервере. , Насколько большим должен быть файл дампа? Что вам нужно знать, так это то, что информация о файле дампа очень примитивна и определенно не подходит для просмотра людьми напрямую, а содержимое, отображаемое jmap -histo, слишком простое, например, оно только показывает, сколько памяти занимают определенные типы объектов и количество этих объектов. , Но нет более подробной информации, например, кто создал эти объекты. Итак, какая польза от файла дампа? Конечно полезно, потому что есть инструмент для анализа файла дампа памяти jvm.
6: анализатор памяти eclipse
S0 S1 E O P YGC YGCT FGC FGCT GCT
54.62 0.00 42.87 43.52 86.24 1792 5.093 33 7.670 12.763
S0 S1 E O P YGC YGCT FGC FGCT GCT
54.62 0.00 42.87 43.52 86.24 1792 5.093 33 7.670 12.763
S0: Зона susvivor0 нового поколения, коэффициент использования площадей 54 . 62%
S1: область susvivor1 нового поколения, коэффициент использования пространства составляет 0,00% (поскольку второй второстепенный сбор не был выполнен)
Java heap space ошибка как исправить
If your application’s execution time becomes longer, or if the operating system seems to be performing slower, this could be an indication of a memory leak. In other words, virtual memory is being allocated but is not being returned when it is no longer needed. Eventually the application or the system runs out of memory, and the application terminates abnormally.
This chapter contains the following sections:
The java.lang.OutOfMemoryError Error
One common indication of a memory leak is the java.lang.OutOfMemoryError error. This error indicates that the garbage collector cannot make space available to accommodate a new object, and the heap cannot be expanded further. This error may also be thrown when there is insufficient native memory to support the loading of a Java class. In rare instances, the error is thrown when an excessive amount of time is being spent performing garbage collection, and little memory is being freed.
The java.lang.OutOfMemoryError error can also be thrown by native library code when a native allocation cannot be satisfied (for example, if swap space is low).
A stack trace is printed when a java.lang.OutOfMemoryError error is thrown.
An early step to diagnose a java.lang.OutOfMemoryError error is to determine its cause. Was it thrown because the Java heap is full or because the native heap is full? To help you discover the cause, a detail message is appended to the text of the exception, as shown in the following examples:
Detail Message: Java heap space
Cause: The detail message Java heap space indicates that an object could not be allocated in the Java heap. This error does not necessarily imply a memory leak. The problem can be as simple as a configuration issue, where the specified heap size (or the default size, if it is not specified) is insufficient for the application. The initial and maximum size of the Java heap space can be configured using the –Xms and –Xmx options.
The APIs that are called by an application could also unintentionally be holding object references.
One other potential source of this error arises with applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. Instead, after garbage collection, the objects are queued for finalization, which occurs at a later time. In the Oracle implementations of the Java Runtime, finalizers are executed by a daemon thread that services the finalization queue. If the thread cannot keep up with the finalization queue, then the Java heap could fill up, and this kind of java.lang.OutOfMemoryError error would be thrown. One scenario that can cause this situation is when an application creates high-priority threads that cause the finalization queue to increase at a rate that is faster than the rate at which the finalizer thread is servicing that queue.
Action: Try increasing the Java heap size. See Monitoring the Objects Pending Finalization to learn more about how to monitor objects for which finalization is pending. See Finalization and Weak, Soft, and Phantom References in Java Platform, Standard Edition HotSpot Virtual Machine Garbage Collection Tuning Guide for information about detecting and migrating from finalization.
Detail Message: GC Overhead limit exceeded
Cause: The detail message GC overhead limit exceeded indicates that the garbage collector (GC) is running most of the time, and the Java application is making very slow progress. After a garbage collection, if the Java application spends more than approximately 98% of its time performing garbage collection and if it is recovering less than 2% of the heap and has been doing so for the last five (compile-time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError error is thrown. This error is typically thrown because the amount of live data barely fits into the Java heap leaving little free space for new allocations.
Action: Increase the heap size. The java.lang.OutOfMemoryError error for GC Overhead limit exceeded can be turned off using the command-line flag -XX:-UseGCOverheadLimit .
Detail Message: Requested array size exceeds VM limit
Cause: The detail message «Requested array size exceeds VM limit» indicates that the application (or APIs used by that application) attempted to allocate an array with a size larger than the VM implementation limit, irrespective of how much heap size is available.
Action: Ensure that your application (or APIs used by that application) allocates an array with a size less than the VM implementation limit
Detail Message: Metaspace
Cause: Java class metadata (the virtual machine’s internal presentation of a Java class) is allocated in native memory (referred to here as Metaspace). If the Metaspace for class metadata is exhausted, a java.lang.OutOfMemoryError error with a detail message Metaspace is thrown. The amount of Metaspace that can be used for class metadata is limited by the parameter MaxMetaSpaceSize , which can be specified on the command line. When the amount of native memory needed for class metadata exceeds MaxMetaSpaceSize , a java.lang.OutOfMemoryError error with detail message Metaspace is thrown.
Action: If MaxMetaSpaceSize has been specified on the command-line, increase its value. Metaspace is allocated from the same address space as the Java heap. Reducing the size of the Java heap will make more space available for Metaspace. This trade-off is only useful if there is an excess of free space in the Java heap. See the following action for the Out of swap space detail message.
Detail Message: request size bytes for reason . Out of swap space?
Cause: The detail message request size bytes for reason . Out of swap space? appears to be a java.lang.OutOfMemoryError error. However, Java reports this apparent error when an allocation from the native heap failed and the native heap might be close to exhaustion. The message indicates the size (in bytes) of the request that failed and the reason for the memory request. Usually the reason is the name of a source module reporting the allocation failure, although sometimes it indicates the actual reason.
Action: When this error is thrown, the Java VM (JVM) invokes the fatal error handling mechanism: it generates a fatal error log file, which contains useful information about the thread, process, and system at the time of the crash. In the case of native heap exhaustion, the heap memory and memory map information in the log can be useful. See Fatal Error Log.
You might need to use troubleshooting utilities for the operating system to diagnose the issue further. See Native Operating System Tools.
Detail Message: Compressed class space
Cause: On 64-bit platforms, a pointer to class metadata can be represented by 32-bit offset (with UseCompressedOops ). This is controlled by the command-line flag UseCompressedClassPointers ( true by default). If UseCompressedClassPointers is true , the amount of space available for class metadata is fixed at the amount CompressedClassSpaceSize . If the space needed for UseCompressedClassPointers exceeds CompressedClassSpaceSize , a java.lang.OutOfMemoryError error with detail message Compressed class space is thrown.
There are bounds on the acceptable size of CompressedClassSpaceSize . For example -XX:CompressedClassSpaceSize=4g , exceeds acceptable bounds and will result in a message such as
There is more than one kind of class metadata: –klass metadata, and other metadata. Only klass metadata is stored in the space bounded by CompressedClassSpaceSize . Other metadata is stored in Metaspace.
Cause: This detail message indicates that a native method, has encountered an allocation failure. The difference between this and the previous message is that the allocation failure was detected in a Java Native Interface (JNI) or native method rather than in the JVM itself.
Action: If this type of java.lang.OutOfMemoryError error is thrown, you might need to use native utilities of the operating system to diagnose the issue further. See Native Operating System Tools.
Detecting a Memory Leak
The java.lang.OutOfMemoryError error can be an indication of a memory leak in a Java application. It might also indicate that either the Java heap, Metaspace or the Compressed Class space is sized smaller than the memory requirements of the application for that specific memory pool. Before assuming a memory leak in the application, ensure that the memory pool for which you are seeing the java.lang.OutOfMemoryError error is sized adequately. The Java heap can be sized using the –Xmx and –Xms command-line options, and the maximum and initial size of Metaspace can be configured using MaxMetaspaceSize and MetaspaceSize . Similarly, the Compressed Class space can be sized using the CompressedClassSpaceSize option.
Memory leaks are often very difficult to detect, especially those that are slow. A memory leak occurs when an application unintentionally holds references to Java objects or classes, preventing them from being garbage collected. These unintentionally held objects or classes can grow in memory over time, eventually filling up the entire Java heap or Metaspace, causing frequent garbage collections and eventual process termination with a java.lang.OutOfMemoryError error.
For detecting memory leaks, it is important to monitor the live set of the application that is, the amount of Java heap space or Metaspace being used after a full garbage collection. If the live set increases over time after the application has reached a stable state and is under a stable load, that could be a strong indication of a memory leak. The live set and memory usage of an application can be monitored by using JConsole and JDK Mission Control . The memory usage information can also be extracted from garbage collection logs.
Note that if the detail message of the error suggests the exhaustion of the native heap, the application could be encountering a native memory leak. To confirm native memory leaks, use native tools such as pmap or PerfMon , and compare their periodically-collected output to determine the newly-allocated or growing memory sections of the process.
JConsole
JConsole is a great tool for monitoring resources of Java applications. Among other things, it is helpful in monitoring the usage of various memory pools of an application, including generations of Java heap, Metaspace, Compressed Class Space, and CodeHeap.
In the following screenshots, for an example program, the JConsole shows the usage of Heap Memory and Old Generation steadily increasing over a period of time. This steady growth in memory usage even after several full garbage collections indicates a memory leak.
Figure 3-1 JConsole Heap Memory
Figure 3-2 JConsole Old Generation
JDK Mission Control
You can detect memory leaks early and prevent java.lang.OutOfMemoryError errors using JDK Mission Control (JMC).
Detecting a slow memory leak can be difficult. A typical symptom could be the application becoming slower after running for a long time due to frequent garbage collections. Eventually, java.lang.OutOfMemoryError errors may be seen. However, memory leaks can be detected early, even before such problems occur, by analyzing Java Flight recordings.
Watch if the live set of your application is increasing over time. The live set is the amount of Java heap being used after a full garbage collection, which collects all the unreachable objects. To inspect the live set, start JMC and connect to a JVM using the Java Management console (JMX). Open the MBean Browser tab and look for the GarbageCollectorAggregator MBean under com.sun.management .
Start JMC and start a Time fixed recording (profiling recording) for an hour. Before starting a flight recording, make sure that the option Object Types + Allocation Stack Traces + Path to GC Root is selected from the Memory Leak Detection setting.
Once the recording is complete, the recording file ( .jfr ) opens in JMC. Look at the Automated Analysis Results page. To detect a memory leak focus on the Live Objects section of the page. Here is an example of a recording, which shows a heap size issue:
Figure 3-3 Memory Leak — Automated Analysis Page
Description of «Figure 3-3 Memory Leak — Automated Analysis Page»
You can observe that in the Heap Live Set Trend section, the live set on the heap seems to increase rapidly and the analysis of the reference tree detected a leak candidate.
For further analysis, open the Java Applications page and then click the Memory page. Here is a sample figure of a recording, which shows memory leak issue.
Figure 3-4 Memory Leak — Memory Page
Description of «Figure 3-4 Memory Leak — Memory Page»
You can observe from the graph that the memory usage has increased steadily, which indicates a memory leak issue.
Garbage Collection logs
Memory usage information can be extracted using GC logs as well. If the GC logs show that the application has performed several full garbage collections attempting to reclaim space in Old generation or Metaspace, but without any significant gain, this indicates that the application might be suffering from a memory leak problem.
This will log messages tagged with at least gc using info level, and messages tagged with exactly gc and phases tags using debug level to a file called gc.log .
Diagnosing Java Memory Leaks
Diagnosing leaks in Java source code can be difficult. Usually, it requires very detailed knowledge of the application. In addition, the process is often iterative and lengthy. This section provides information about the tools that you can use to diagnose memory leaks in Java source code.
Diagnostic Data
This section looks at the diagnostic data you can use to troubleshoot memory leaks.
Heap Histograms
You can try to quickly narrow down a memory leak by examining the heap histogram. You can get a heap histogram in several ways:
- If the Java process is started with the -XX:+PrintClassHistogram command-line option, then the Control+Break handler will produce a heap histogram.
- You can use the jmap utility to get a heap histogram from a running process:
It is recommended to use the latest utility, jcmd , instead of jmap utility for enhanced diagnostics and reduced performance overhead. See Useful Commands for the jcmd Utility. The command in the following example creates a heap histogram for a running process using jcmd and results similar to the following jmap command.
The output shows the total size and instance count for each class type in the heap. If a sequence of histograms is obtained (for example, every two minutes), then you might be able to see a trend that can lead to further analysis.
For example, if you specify the -XX:+CrashOnOutOfMemoryError command-line option while running your application, then when a java.lang.OutOfMemoryError error is thrown, the JVM will generate a core dump. You can then execute jhsdb jmap on the core file to get a histogram, as shown in the following example.
The above example shows that the java.lang.OutOfMemoryError error was caused by the number of byte arrays (2108 instances in the heap). Without further analysis it is not clear where the byte arrays are allocated. However, the information is still useful.
Heap Dumps
Heap dumps are the most important data to troubleshoot memory leaks. Heap dumps can be collected using jcmd , jmap , JConsole tools, and the -XX:+HeapDumpOnOutOfMemoryError Java option
-
You can use the GC.heap_dump command with the jcmd utility to create a heap dump as shown below:
Java Flight Recordings
Figure 3-5 Flight Recording Template Manager
The flight recordings can then be created using any of the following ways:
- Java Flight Recorder options
Class Loader Statistics
Information about class loaders and the number of classes loaded by them can be very helpful in diagnosing memory leaks related to Metaspace and Compressed Class Space.
- jcmd <process id/main name> VM.classloader_stats
- jmap –clstats <process id>
The following is an example of output generated by jmap .
Analysis Tools
This section explores the analysis tools you can use to diagnose memory leaks, including those that can analyze diagnostic data described above.
Heap Dump Analysis Tools
There are many third-party tools available for heap dump analysis. The Eclipse Memory Analyzer Tool (MAT) and YourKit are two examples of commercial tools with memory debugging capabilities. There are many others, and no specific product is recommended.
JDK Mission Control (JMC)
The Flight Recorder records detailed information about the Java runtime and Java applications running on the Java runtime.
This section describes how to debug a memory leak by analyzing a flight recording in JMC.
You can use the Java Flight Recordings to identify the leaking objects. To find the leaking class, open the Memory page and click the Live Objects page. Here is a sample figure of a recording, which shows the leaking class.
Figure 3-6 Memory Leak — Live Objects Page
Description of «Figure 3-6 Memory Leak — Live Objects Page»
You can observe that most of the live objects being tracked are held by Leak$DemoThread , which in turn holds a leaked char[] class. For further analysis, see the Old Object Sample event in the Results tab that contains sampling of the objects that have survived. This event contains the time of allocation, the allocation stack trace, and the path back to the GC root.
When a potentially leaking class is identified, look at the TLAB Allocations page in the JVM Internals page for some samples of where objects were allocated. Here is a sample recording, showing TLAB allocations.
Figure 3-7 Memory Leak — TLAB Allocations
Description of «Figure 3-7 Memory Leak — TLAB Allocations»
Check the class samples being allocated. If the leak is slow, there may be a few allocations of this object and may be no samples. Also, it may be that only a specific allocation site is leading to a leak. You can make required changes to the code to fix the leaking class.
The jfr tool
Java Flight Recorder (JFR) records detailed information about the Java runtime and the Java application running on the Java runtime. This information can be used to identify memory leaks.
To detect a memory leak, JFR must be running at the time that the leak occurs. The overhead of JFR is very low, less than 1%, and it has been designed to be safe to have always on in production.
Start a recording when the application is started using the java command as shown in the following example:
When the JVM runs out of memory and exits due to a java.lang.OutOfMemoryError error, a recording with the prefix hs_oom_pid is often, but not always, written to the directory in which the JVM was started. An alternative way to get a recording is to dump it before the application runs out of memory using the jcmd tool, as shown in the following example:
When you have a recording, use the jfr tool located in the java-home /bin directory to print Old Object Sample events that contain information about potential memory leaks. The following example shows the command and an example of the output from a recording for an application with the pid 16276:
To identify a possible memory leak, review the following elements in the recording:
First, notice that the lastKnownHeapUsage element in the Old Object Sample events is increasing over time, from 63.9 MB in the first event in the example to 121.7 MB in the last event. This increase is an indication that there is a memory leak. Most applications allocate objects during startup and then allocate temporary objects that are periodically garbage collected. Objects that are not garbage collected, for whatever reason, accumulate over time and increase the value of lastKnownHeapUsage .
Next, look at the allocationTime element to see when the object was allocated. Objects that are allocated during startup are typically not memory leaks, neither are objects allocated close to when the dump was taken. The objectAge element shows how long the object has been alive. The startTime and duration elements are not related to when the memory leak occurred, but when the OldObject event was emitted and how long it took to gather data for it. This information can be ignored.
Then look at the object element to see the memory leak candidate; in this example, an object of type java.util.HashMap$Node . It is held by the table field in the java.util.HashMap class, which is held by java.util.HashSet , which in turn is held by the users field of the Application class.
The root element contains information about the GC root. In this example, the Application class is held by a stack variable in the main thread. The eventThread element provides information about the thread that allocated the object.
If the application is started with the -XX:StartFlightRecording:settings=profile option, then the recording also contains the stack trace from where the object was allocated, as shown in the following example:
In this example we can see that the object was put in the HashSet when the storeUser(String, String) method was called. This suggests that the cause of the memory leak might be objects that were not removed from the HashSet when the user logged out.
It is not recommended to always run all applications with the -XX:StartFlightRecording:settings=profile option due to overhead in certain allocation-intensive applications, but is typically OK when debugging. Overhead is usually less than 2%.
Setting path-to-gc-roots=true creates overhead, similar to a full garbage collection, but also provides reference chains back to the GC root, which is usually sufficient information to find the cause of a memory leak.
NetBeans Profiler
The NetBeans Profiler can locate memory leaks very quickly. Commercial memory leak debugging tools can take a long time to locate a leak in a large application. The NetBeans Profiler, however, uses the pattern of memory allocations and reclamations that such objects typically demonstrate. This process includes also the lack of memory reclamations. The profiler can check where these objects were allocated, which often is sufficient to identify the root cause of the leak.
Diagnosing Native Memory Leaks
Several techniques can be used to find and isolate native code memory leaks. In general, there is no ideal solution for all platforms.
The following are some techniques to diagnose leaks in native code.
Tracking All Memory Allocation and Free Calls
A very common practice is to track all allocation and free calls of native allocations. This can be a fairly simple process or a very sophisticated one. Many products over the years have been built up around the tracking of native heap allocations and the use of that memory.
Tools such as IBM Rational Purify can be used to find these leaks in normal native code situations and also find any access to native heap memory that represents assignments to uninitialized memory or accesses to freed memory.
Not all these types of tools will work with Java applications that use native code, and usually these tools are platform-specific. Because the JVM dynamically creates code at runtime, these tools can incorrectly interpret the code and fail to run at all, or give false information. Check with your tool vendor to ensure that the version of the tool works with the version of the JVM you are using.
See SourceForge for many simple and portable native memory leak detecting examples. Most libraries and tools assume that you can recompile or edit the source of the application and place wrapper functions over the allocation functions. The more powerful of these tools allow you to run your application unchanged by interposing over these allocation functions dynamically.
Native memory leaks can result from native allocations performed either internally by the JVM, or from outside the JVM. The following two sections discuss in detail how both of these memory leaks can be diagnosed.
Native Memory Leaks for Allocations performed by the JVM
The JVM has a powerful tool called Native Memory Tracking (NMT) that tracks native memory allocations performed internally by the JVM. Note that this tool cannot track native memory allocated outside of the JVM, for example by JNI code.
- Enable NMT in the process that you want to monitor by using the Java option NativeMemoryTracking . The output level of the tracking can be set to a summary or detail level as shown below:
- The jcmd tool can then be used to attach to the NMT-enabled process, and obtain its native memory usage details. It is also possible to collect a baseline of memory usage and then collect the difference in usage against that baseline.
Enabling NMT can result in a performance drop of around 5 to 10 percent. Therefore, it should be enabled in production systems cautiously. Additionally, the native memory used by NMT is tracked by the tool itself.
Figure 3-8 JConsole Compressed Class Space
Native Memory Leaks from Outside the JVM
For native memory leaks originating outside the JVM, you can use platform native or other third-party tools for their detection and troubleshooting. Here is a list of some of the tools that you can find useful in troubleshooting native memory leaks caused by allocations performed outside the JVM.
- Valgrind
- Purify available on both UNIX platforms and Windows
- Using Crash Dump or Core files
- On Windows, search Microsoft Docs for debug support. The Microsoft C++ compiler has the /Md and /Mdd compiler options that will automatically include extra support for tracking memory allocation. User-Mode Dump Heap (UMDH) is also helpful in tracking memory allocations.
- Linux systems have tools such as mtrace and libnjamd to help in dealing with allocation tracking.
The following section takes a closer look at two of these tools.
A suppression file can be supplied to valgrind with —log-file option in order for it to not consider the JVM internal allocations (such as the Java heap allocation) as potential memory leaks, otherwise it becomes very difficult to parse through the verbose output and manually look for relevant leak reports.
In the above output, Valgrind correctly reports that there are allocations leaking from Java_java_util_zip_Deflater_init native method.
Using Valgrind may have a negative impact on the performance of the monitored application.
Crash Dump or Core files
On UNIX platforms, the pmap tool is helpful in identifying the memory blocks that might be changing/growing in size over time. Once you have identified the growing memory blocks or sections, you can examine the corresponding crash dump or core file(s) to look at those memory blocks. The values and contents at those locations can provide some valuable clues, which can help tie them back to the source code responsible for the allocations in those memory blocks.
From the previous pmap output, we can see that the memory block at 00007f6d690a3000 is growing between the two memory snapshots of the process. Using a core file collected from the process, we can examine the contents of this memory block.
The above shows that there is a repeating string, «Alert: JNI Memory Leak», present in that memory block. Searching in the source code for the string or contents found in the relevant memory block can lead us to the culprit in the code. Here is the code used for this example, where these allocations are performed in JNI code and are not being released.
Hence, the pmap tool and core files can help in getting to the root of native memory leaks caused by allocations performed outside of the JVM.
Tracking All Memory Allocations in the JNI Library
If you write a JNI library, then consider creating a localized way to ensure that your library does not leak memory, by using a simple wrapper approach.
The procedure in the following example is an easy localized allocation tracking approach for a JNI library. First, define the following lines in all source files.
Then, you can use the functions in the following example to watch for leaks.
The JNI library would then need to periodically (or at shutdown) check the value of the total_allocated variable to verify that it made sense. The preceding code could also be expanded to save in a linked list the allocations that remained, and report where the leaked memory was allocated. This is a localized and portable way to track memory allocations in a single set of sources. You would need to ensure that debug_free() was called only with the pointer that came from debug_malloc() , and you would also need to create similar functions for realloc() , calloc() , strdup() , and so forth, if they were used.
A more global way to look for native heap memory leaks involves interposition of the library calls for the entire process.
Monitoring the Objects Pending Finalization
Different commands and options available to monitor objects pending finalization.
When the java.lang.OutOfMemoryError error is thrown with the «Java heap space» detail message, the cause can be the excessive use of finalizers. To diagnose this, you have several options for monitoring the number of objects that are pending finalization:
- The JConsole management tool can be used to monitor the number of objects that are pending finalization. This tool reports the pending finalization count in the memory statistics on the Summary tab pane. The count is approximate, but it can be used to characterize an application and understand if it relies heavily on finalization.
- On Linux, the jmap utility can be used with the -finalizerinfo option to print information about objects awaiting finalization.
- An application can report the approximate number of objects pending finalization using the getObjectPendingFinalizationCount method of the java.lang.management.MemoryMXBean class. Links to the API documentation and example code can be found in Custom Diagnostic Tools. The example code can easily be extended to include the reporting of the pending finalization count.
Troubleshooting a Crash Instead of a java.lang.OutOfMemoryError error
Use the information in the fatal error log or the crash dump to troubleshoot a crash.
Sometimes an application crashes soon after an allocation from the native heap fails. This occurs with native code that does not check for errors returned by the memory allocation functions.
For example, the malloc system call returns null if there is no memory available. If the return from malloc is not checked, then the application might crash when it attempts to access an invalid memory location. Depending on the circumstances, this type of issue can be difficult to locate.
However, sometimes the information from the fatal error log or the crash dump is sufficient to diagnose this issue. The fatal error log is covered in detail in Fatal Error Log. If the cause of the crash is an allocation failure, then determine the reason for the allocation failure. As with any other native heap issue, the system might be configured with an insufficient amount of swap space, another process on the system might be consuming all memory resources, or there might be a leak in the application (or in the APIs that it calls) that causes the system to run out of memory.
Какие бывают типы OutOfMemoryError или из каких частей состоит память java процесса
Если вы словили OutOfMemoryError, то это вовсе не значит, что ваше приложение создает много объектов, которые не могут почиститься сборщиком мусора и заполняют всю память, выделенную вами с помощью параметра -Xmx. Я, как минимум, могу придумать два других случая, когда вы можете увидеть эту ошибку. Дело в том, что память java процесса не ограничивается областью -Xmx, где ваше приложение программно создает объекты.
Область памяти, занимаемая java процессом, состоит из нескольких частей. Тип OutOfMemoryError зависит от того, в какой из них не хватило места.
1. java.lang.OutOfMemoryError: Java heap space
Не хватает место в куче, а именно, в области памяти в которую помещаются объекты, создаваемые программно в вашем приложении. Размер задается параметрами -Xms и -Xmx. Если вы пытаетесь создать объект, а места в куче не осталось, то получаете эту ошибку. Обычно проблема кроется в утечке памяти, коих бывает великое множество, и интернет просто пестрит статьями на эту тему.
2. java.lang.OutOfMemoryError: PermGen space
Данная ошибка возникает при нехватке места в Permanent области, размер которой задается параметрами -XX:PermSize и -XX:MaxPermSize. Что там лежит и как бороться с OutOfMemoryError возникающей там, я уже описал подробнейшим образом тут.
3. java.lang.OutOfMemoryError: GC overhead limit exceeded
Данная ошибка может возникнуть как при переполнении первой, так и второй областей. Связана она с тем, что памяти осталось мало и GC постоянно работает, пытаясь высвободить немного места. Данную ошибку можно отключить с помощью параметра -XX:-UseGCOverheadLimit, но, конечно же, её надо не отключать, а либо решать проблему утечки памяти, либо выделять больше объема, либо менять настройки GC.
4. java.lang.OutOfMemoryError: unable to create new native thread
Впервые я столкнулся с данной ошибкой несколько лет назад, когда занимался нагрузочным тестированием и пытался выяснить максимальное количество пользователей, которые могут работать с нашим веб-приложением. Я использовал специальную тулзу, которая позволяла логинить пользователей и эмулировать их стандартные действия. На определенном количестве клиентов, я начал получать OutOfMemoryError. Не особо вчитываясь в текст сообщения и думая, что мне не хватает памяти на создание сессии пользователя и других необходимых объектов, я увеличил размер кучи приложения (-Xmx). Каково же было мое удивление, когда после этого количество пользователей одновременно работающих с системой только уменьшилось. Давайте подробно разберемся как же такое получилось.
На самом деле это очень просто воспроизвести на windows на 32-битной машине, так как там процессу выделяется не больше 2Гб.
Допустим у вас есть приложение с большим количеством одновременно работающих пользователей, которое запускается с параметрами -Xmx1024M -XX:MaxPermSize=256M -Xss512K. Если всего процессу доступно 2G, то остается свободным еще коло 768M. Именно в данном остатке памяти и создаются стеки потоков. Таким образом, примерно вы можете создать не больше 768*(1024/512)=1536 (у меня при таких параметрах получилось создать 1316) нитей (см. рисунок в начале статьи), после чего вы получите OutOfMemoryError. Если вы увеличиваете -Xmx, то количество потоков, которые вы можете создать соответственно уменьшается. Вариант с уменьшением -Xss, для возможности создания большего количества потоков, не всегда выход, так как, возможно, у вас существуют в системе потоки требующие довольно больших стеков. Например, поток инициализации или какие-нибудь фоновые задачи. Но все же выход есть. Оказывается при программном создании потока, можно указать размер стека: Thread(ThreadGroup group, Runnable target, String name,long stackSize). Таким образом вы можете выставить -Xss довольно маленьким, а действия требующие больших стеков, выполнять в отдельных потоках, созданных с помощью упомянутого выше конструктора.
Более подробно, что же лежит в стеке потока, и куда уходит эта память, можно прочитать тут.
Конечно, вам может показаться данная проблема слегка надуманной, так как большинство серверов нынче крутиться на 64-битной архитектуре, но все же считаю данный пример весьма полезным, так как он помогает разобраться из каких частей состоит память java-процесса.
OutOfMemory Java Heap Space
From the diagram above, we can see that the heap is used to allocate memory to Objects and JRE classes.
The heap is created when the JVM starts up and may increase or decrease in size while the application runs.
Heap Size
Java applications are only allowed to use a limited amount of memory.
Heap size is set during the JVM launch and can be modified by specifying JVM parameters -Xmx .
For example, in setenv.bat of tomcat,
If you do not explicitly set the sizes, platform-specific defaults will be used.
OutOfMemory — Java Heap Space
When the heap becomes full, garbage is collected.
During the garbage collection objects that are no longer used/referenced are cleared, thus making space for new objects.
Hence, if JVM cannot allocate an object because heap is out of memory, and no more memory could be made available by the garbage collector, then OutOfMemory is thrown.
Reasons
There are two main reasons to get this error:
poor programming cause memory leak — infinite loop, not clearing memory by closing resources etc
low heap size — java is running on less memory than required
Memory Leak
Garbage collection ( GC) is the process to remove unreferenced objects from the heap memory. However GC may not always give you 100% results. There might be some unused objects which GC cannot remove because some other object is still holding a reference to it. This concept is called as Memory Leak.
We can see in the below image that object X is holding the reference of object Y. Object Y has completed its task but cannot be garbage collected because of the unwanted reference.
How to solve
increase the heap size
find the root cause to memory leak in your programme and solve it
In most cases, #2 is what we should do. If there is memory leak in your code, only increase heap size cannot eliminate this error.
In order to find the memory leak in code, we need to :
Get the heap dump info
Analysise the heap dump
Get the heap dum info
manually collect
jmap is a JDK built in tool which can be used to create heap dump file by using the following command:
jmap -heap:live,format=b,file=/home/test.bin <pid>
<pid> is the process id of java.
It’s quite important to pass live option. If this option is passed, then only live objects in the memory are written into the heap dump file.
If this option is not passed, all the objects, even the ones which are ready to be garbage collected are printed in the heap dump file. It will increase the heap dump file size significantly as well as analysis tedious.
automatic collect
Alternatively, you can add -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=D:\heapdump.bin to tomcat CATALINA_OPTS config, such that tomcat will create heap dump file when it encounters OutOfMemory error.
Analysise the heap dump
The heap dump file can be parsed by jvisualvm (which is jdk built in tool) or eclipse memory analysis(it is available as a stand-alone application or as a plugin for Eclipse).
Eclipse Memory Analysis Tool
The MAT looks like next:
The top left button with circled by red pen is:Accquire heap dump from a locally running vm, its next button is:open heap dump
Shallow vs Retained Heap
Shallow heap is the memory consumed by one object. This object might be referencing to other objects in the heap memory. Retained heap is sum of memory of this object’s and all other referenced objects.
Any object X might be consuming small memory by itself but referencing too many other unused objects. If this object is alive, garbage collector will not be able to free these unused objects from memory. In this case object X becomes important to be garbage collected because it is keeping a large set of retained heap. User should find and debug the objects where shallow heap is low and retained heap is high.
Dominator tree
There is a OutOfMemory error — java heap space thrown, and its heap dump info looks like:
From the above screenshot, we can see that the biggest value of retained heap is about the jdbcresult, so the memory leak is caused by a sql query.
Right click this entry -> List Objects -> with outgoing references, we can see that:
The above screen shot tells us that this sql query has too many results, as a consequence, it cause the OutOfMemory error.