In my experience, it's typical for Domino to hold onto memory it has allocated even when it no longer needs it.
Domino has a "memory manager" process which (I think) parcels out memory to its other processes, and keeps allocated memory to avoid frequent deallocations and reallocations.
I have recently been reviewing RAM usage for one customer that has a few Windows servers replicating some large databases, with up to 2 million docs and 7 GB file size per NSF.
One server with 8 GB RAM is basically just for Domino, and doesn't exceed 5 GB RAM in use even after weeks of continuous operation.
Another server with 32 GB RAM is running Domino but is also used as a file server and maybe some other tasks, and doesn't exceed 8 GB RAM in use even after weeks of continuous operation.
These two servers are both 64-bit Windows and 32-bit Domino.
I've never dealt with 64-bit Domino, but wouldn't be surprised if it used up to double the RAM of 32-bit.
How much total RAM does your server have?
IBM Domino server hogging RAM (Nad Surf 31.May.18)
Feedback response number SLESAZBB2X created by Scott Leis on 06/01/2018
. . Questions and comments... (Scott Leis 1.Jun.18)
. . . . amount of RAM (Nad Surf 1.Jun.18)
. . What facilities are in use? (Mike Woolsey 4.Jun.18)
. . . . Issue resolved. (Nad Surf 23.Jul.18)