As a Domino administrator, you can monitor the Domino server(s) for error messages that indicate that the Java Virtual Machine (JVM) is out of memory; which can directly impact the HTTP process or the Domino server overall. There are several types of messages written to the Domino console that can be set up with alerts to notify admins once they occur. Those messages are:
HTTP Web Server: Out Of Memory Exception
HTTP JVM: java.lang.OutOfMemoryError
HTTP JVM: OutOfMemoryError
Beginning in Domino 8.5.x the HTTP JVM size has been decoupled from the JavaMaxHeapSize ini parameter, which is applied to all other Domino server processes that load the JVM and can be controlled separately. Since JavaMaxHeapSize sets the java heap for all other Domino server processes that load a JVM, for example, Agent Manager, Runjava, etc., then an Agent Manager process which periodically executes java agents will inherit the JVM heap size set in JavaMaxHeapSize. HTTPJVMMaxHeapSize sets the JVM heap only for the HTTP process.
Common causes for the above "out of memory" errors/exceptions is for HTTP JVM memory exhaustion when running Java agents, XPages, or servlets.
Actions to take
1. Restart the HTTP process. If the problem is isolated only to HTTP private memory, perhaps recycling the HTTP process will allow memory to be reclaimed and available upon restart.
tell http quit
load http
2. Ensure that the Java and/or JVM Heap Size is set appropriately. JavaMaxHeapSize applies pre-8.5 Domino and should be reviewed in pre-8.5 environments. JavaMinHeapSize is also still valid for Domino 7.x and 8.0.1 servers.
Check your notes.ini for HTTPJVMMaxHeapSize. This value may need to be altered depending on Domino features enabled.
3. Ensure your application or agent is efficiently performing garbage collection
Monitor memory usage
1. You can use the runtime statistics to ensure your application is not leaking memory
2. You can monitor the memory Heap by adding the following code to your application:
Runtime s_runtime = Runtime.getRuntime ();
System.out.println(String.valueOf(s_runtime.totalMemory() - s_runtime.freeMemory()))
Be aware that Runtime.getRuntime().totalMemory() will not return 256 Mb if "JavaMaxHeapSize=268435456/256Mb" was set in the Notes.ini. It will grow as more memory is required and will return the OutOfMemory error or hang once it reaches the max threshold of 256 Mb.
Before attempting to collect a memory dump, you should ensure that you are facing a heap exhaustion. You can use JavaVerboseGC=1 to monitor this and how well objects are garbage collected.
3. In 8.5.3 and higher, you can enable a program document in conjunction with the new command to generate heap dumps to monitor the Java heap size.
tell http dump java heap
Data Collection
A. Enable debug for Java agents DebugShowJavaAgentTid=1 and Debug_Agent_Thread=1
B. Enable debug for Java servlets
C. Collect Heap dumps for xPages related issues.
Starting with Domino 8.5.1, XSP component supports the following Domino console command to generate the size of the Java heap:
Upon completion, the command would respond with the generated Java heap dump location:
tell http xsp heapdump
[086C:0002-1F70] 05-04-2011 10:40:47 AM HTTP JVM: JVMDUMP007I JVM Requesting Heap dump using 'C:\Lotus\Domino85164Bits\heapdump.20110405.104047.2156.0001.phd'
[086C:0002-1F70] 05-04-2011 10:40:48 AM HTTP JVM: JVMDUMP010I Heap dump written to C:\Lotus\Domino85164Bits\heapdump.20110405.104047.2156.0001.phd
Starting in 8.5.3, the http server supports some java commands that allow Domino to dump jvm information like core, heap, system. The commands are as follows:
tell http dump java heap
tell http dump java core
tell http dump java system
tell http dump java * - dumps both core and heap
Note: It may take several minutes for the commands to be fully processed.
Recommendations
For web applications that require a large amount of memory to service requests, it is recommended that you utilize 64-bit Domino (available on AIX and Windows) for a larger memory footprint.
Related Documents