This article will describe areas in an NSD log to gather information on memory related crash on HTTP process.
Memory crash Memory exhaustion defined by “Insufficient memory” messages
Memory Handles defined by “Out of Handles” messages
java.lang.OutOfMemory errors
NSD log contains memory summary for Domino processes. Start by reviewing the Process Memory Mappings section for HTTP process. For NSDs generated from Unix OS, use “Process Heap” size under “Process Heap Memory Stats” instead.
In the example below, Process Memory Mapping shows that HTTP private memory is over 1G in size:
HTTP overall memory is comprised of the following areas:
1. Private Static-DPools
2. MBlocks
3. Thread stack Space
4. JVM
5. LotusScript
Next step is to identify which of these areas is making up majority of HTTP overall memory.
Private Static-DPools
Memory allocations performed by Notes using various DPools (4 MB to 8 MB). These pools are not freed back to the OS after being used.
NSD or Memcheck log look at “Process Heap Memory Stats”
DPools are found within an NSD or Memcheck log under “Process Heap Memory Stats” section:
Check Top 10 section for HTTP in NSD log to identify any memory blocks that are large in size or have high handle count.
DPool debug for HTTP process will generate call stacks in Console log when DPool is allocated. Note, DPool debug can also be used for Shared memory which does not apply in this case.
Example of DPool debug for HTTP on Windows OS uses 'nhttp' as value. In this example, debug will track DPool allocations for memory block 0x044
debug_private_pools=1
debug_private_pools_process_name=nhttp
debug_private_pools_blocktype=044f
Memcheck logs can be collected to track memory usage and identify time frame memory spike occurs or correlate high memory usage with activity on server. The memcheck logs can be automated by setting up Program document to run "-memcheck -perf" over period of time.
MBlocks
MBlocks are memory blocks allocated directly from the OS and released back to the OS when freed by the caller.
From NSD log search for “Directly Allocated Memory” under memcheck for HTTP process.
Thread stack Space
Generally, each HTTP thread uses about 1 MB of memory per thread.
NSD log has a Server Statistics section that includes HTTP.workers stat showing the number of HTTP worker threads configured on the server.
<@@
Notes Data -> Server Data -> Server Statistics (Time 09:20:29)
@@>
...
Http.Workers = 140
...
JVM
Java heap size by default is 64MB however, this can be modified via Notes.ini parameter JavaMaxHeapSize or HTTPJVMMaxHeapSize in Domino 8.5.1 or later. Overall HTTP memory will be impacted in cases when Java heap size has been increased to high number.
NSD log has Notes.ini section to check if these parameters are being used on Server.
<@@
System Data -> Environment -> C:\Lotus\Domino\notes.ini (Time 09:20:58)
@@>
...
LotusScript
LotusScript memory currently found in a memory.dmp file. The “HTTP” process section has “LotusScript Memory Usage” section
