Domino Deployment Best Practices

Table of Contents | Previous Page | Next Page

---

The focus of this chapter is to outline lessons learned from existing deployments.

Efficiently managing your templates

An essential planning step for deploying Domino 8.5, as was the case for prior versions of IBM Lotus Notes and Domino, is to review your template deployment and maintenance strategy.

Managing your templates is important because a lot of the new functionality in a new major release of Domino is due to new design elements that are pushed from template to system databases.

Here are some of the more common methods of Domino template management:

• Centrally managing templates from one Domino server or a subset of servers
• Modifying the ACL to grant limited rights to the servers and potential users
• Modifying the template name inheritance, replica ID, and template file name
• Disabling the replication of all the templates
• Removing all templates from the Domino server and removing the nightly run of the design task

You should use one or more of the above methods to manage your templates. For more information about template strategy, including details on how to create your own template package, reference the Lotus Notes and Domino 7 Enterprise Upgrade Best Practices Redpaper at the following URL:

http://www.redbooks.ibm.com/abstracts/redp4120.html?Open

The method used to manage your templates depends on what you and your Domino administration team have found to work in the past in your environment and how tightly you want to manage and protect your templates in the future Note: If you have customized any of the pre-Domino 8.5 templates, such as the mail template, make sure to thoroughly test your modifications on the new Domino 8.5 templates before deploying the updated templates to your Notes mail files or database applications.

In order to take advantage of the new design features in Domino 8.5, we recommend making customizations to the new Domino 8.5 templates, rather than using back-leveled templates.

As the Domino administrator, managing both the changes to and distribution of your templates needs to be one of your principal concerns during the Domino upgrade process.

Efficiently Upgrading the On Disk Structure ODS

Compact is a highly resource intensive task, depending on the servers I/O performance, it can be benefitial to run several compact tasks in parallel. To avoid those tasks conflicting with each other, its recommended to create indirect files (*.IND).

Indirect files (*.IND) are nothing else than plain text files, containing the path + file name of the databases in scope.

• Log on to the server’s console
• Use the following operating system command to output a list of NSF files into a plain text file.

         dir X:\Domino\Data\*.nsf /s /b >X:\Domino\Data\files.ind

• Based on the example above, a new file ( files.ind )has been created in the directory X:\Domino\Data\
  Now use an editor to parse this file into pieces. We recommend to create one IND file less than the number of CPUs in your server. (for example. three if you have a server with four CPUs).
  

For example:

files1.ind:

X:\Domino\Data\names.nsf
X:\Domino\Data\catalog.nsf
files2.ind:
X:\Domino\Data\mail21\mail1101.nsf
X:\Domino\Data\mail21\mail1102.nsf
X:\Domino\Data\mail22\mail1103.nsf
files3.ind:
X:\Domino\Data\mail23\mail1301.nsf
X:\Domino\Data\mail23\mail1302.nsf
X:\Domino\Data\mail23\mail1303.nsf name of your choice, but make sure the extension is called .ind.

• Now you can run compact against the IND files created above. So with three IND files, you could run 3 compact processes concurrently by
• (n)compact files1.ind –c
  (n)compact files2.ind –c
  (n)compact files3.ind -c
  

Note: Each compact task will process the list of files one by one, so it may take a while to finish!

Optimizing your configuration

A large number of configuration parameters can be used to optimize the server for individual needs. Although all options are explained in various technotes its hard to decide weather or not a configuration parameter should be used. Guidance is provided by using the Domino Configuration Tuner.

Run DCT directly after upgrading your server to check the current configuration. Furthermore it’s recommended to run DCT from time to time, for example, once per month, to constantly apply latest best practices.

More details about the Domino Configuration Tuner itself can be found on
http://www-10.lotus.com/ldd/dominowiki.nsf/dx/domino-configuration-tuner

Storage Optimization

Lotus Domino 8.5 has been designed for providing lowest total cost of ownership. Features such as DAOS, Design and/or attachment compression will lower the disk space requirements for your servers.

However, simply switching on all features is not a guarantee to be the perfect solution for every environment. The following list provides some hints of what you need to think of to get the most out of your servers.

Compacting and White Space

When COMPACT -b runs on a mail file, or indeed any DB; rather than decrease the size of the DB, this Lotus recommended "purge" increases white space and decreases the percentage of the file used. Simply put, compact -b reorders data in a database so that the data is consecutive. The utilization percentage goes up while space is compacted within the NSF structure. The file size remains the same.

By defragmenting the DB, the white spaces become a single large "white space" and thus we now have room for larger incoming items, mail, attachments, and so on. This new incoming "data" we can now accommodate within the existing DB without having to "expand" the DB by adding additional new blocks to the end. The DB will gradually again becomes more fragmented through use. As random records (notes) are deleted creating smaller white space holes, the DB becomes increasingly fragmented. Consequently larger incoming notes and files, attachments and so on, will no longer fit within any spaces inside the DB, and so the DB must expand itself.

Thus frequent defragmentation is required to keep the white space in a usable, contiguous form as much as possible.

Re Using White Space

Because Notes reuses white space within a database before allocating additional storage, the rate of perceived growth of any DB will slow down, and eventually stop, as the amount of white space created by COMPACT -b approaches the actual storage required for new content delivered during the day or days between compactions, and the more or less equal rate of unwanted mail item deletions! This can be called "steady state".

Thus eventually all DBs will reach steady state, freeing and re-using their own white-space, and taking the laborious task of managing and worrying about data growth away from the Administrator. Which is why we recommend it.

Looking closer at the actual OS side of things:

Looking at the situation from an OS file system point of view, this also makes total sense. If we have to expand a file, we have to onerously find free blocks on the disk, map and allocate them to our file, alter the disk index tables (for example, FATs or Inodes), and finally move the file end pointer to the (now extended) end of file block. This is quite an expensive operation, and when repeated multiple times (each time you grow the file) we will start to add to the "dead weight" of non-useful operations you are asking the system to perform.

Disk Fragmentation! A thing of the past?

Administrators tend to ignore disk fragmentation, but shouldn't. Yes disks are much faster, and can recover blocks from anywhere on the surface much quicker these days, but if all the blocks for a file were kept together, rather than randomly scattered, then seek and access times would decrease noticeably.

This is what COMPACT -b attempts to do by periodically using COMPACT -B or -C and making the OS run all over the disk to find additional spare blocks.

Gradually, with continual truncation of white space and the subsequent inevitable re-allocation of new disk blocks, the whole disk will become more and more fragmented, resulting in a gradual, almost imperceptible loss of performance day by day, but over a longer period this will be significant, if hard to measure comparatively. Until you do an OS level disk de-frag and then marvel at the improved performance.

But surely I will need more disk?

No, actually you probably won't. If you think about it, the space just before the compact -C or -B is probably approaching steady state, and soon would be!!

Unless you let users grow their mail files to silly sizes, then they will, as a group probably have steady state on mail received, sent, deleted and stored. But, If you don't defragment the DBs with COMPACT -b every few days, the DBs will grow simply because there are not enough CONTIGUOUS blocks.

If you ran COMPACT -b nightly (if possible) for a week or two you would be pleasantly surprised that, rather than seeing the disk reaching that full point on the Friday before your usual COMPACT -B, it would peak and then achieve steady state, probably below your previously noted high water marks.

What we are saying is that the massive fragmentation of your mail files over a week is what causes you to have such apparent size problems. The continual gentle on-line defragmentation will achieve steady state below or close to your previous watermarks.

Another bonus is that Compact -b can be run on-line without affecting live mail or DB users (it might go a tad slower for a minute or two, but no shutdown is required). Obviously, it is best to run it over-night or at quiet times. You may even want to run it more than once per day. Remember, the more efficient the DB structure, the faster the system can find and serve up the data, and the lower the over-all OS system loading will be.

As another bonus your users will be pleasantly surprised that mail access feels quicker every day of the week, instead of just on Monday morning, because the data has been re-sequenced within the DB, and can now be accessed more quickly, so maybe you can even add a few more users.

Summary

Dont use compact with options –c or –B, as this will increase disk fragmentation, better use compact with lowercase b as option.

You could always do a COMPACT -B every month or two if it makes you feel better, and re-allocate the DBIIDs. However, you will have achieved steady state long before then, and the nightly compact is keeping things very sweet and not intentionally fragmenting your disks.

Transaction Log Optimization

Starting with Domion 8.5 you can optionally use the following Notes.ini parameter to align the block size of your transaction logs to your OS.

CREATE_R85_LOG=1

More details are explained in the following URL:
http://www-10.lotus.com/ldd/dominowiki.nsf/dx/create_r85_log

DAOS and Attachment compression

We already know DAOS is a great space saver, but when configured in the wrong way you will not get all the benefits you could have.

Databases can be enabled for LZ1 compression (“Compress Attachments”) to save disk space. If this option is used in combination with DAOS, the file will first be LZ1 compressed before its written to disk.

To get the most savings, administrators need to ensure that ALL databases on the server use the same DB settings.

Example: Assuming two applications or mail files on the same server store the same attachment.
If one DB is configured to use LZ1 compression and the other is not, DAOS would save the attachment TWICE on disk – one attachment from the DB using LZ1 compression, and another version from the attachment without LZ1 compression.

Summary
When setting up DAOS in your environment, make sure that all databases involved are using the same configuration for compression.

If you experience this scenario, recompressing your databases with the server command “load compact -c –ZU” will resolve this issue.

Mail Template Upgrade

Use one of the following options to upgrade the Domino Mail templates:

• Mail Conversion Utility (issue load convert from the Domino Console).
• Select File -> Database -> Replace Design (from the Notes, Administrator, or Designer client).

To upgrade mail files to a different template using the load convert utility, use the following syntax:

load convert [-r] [-u] filepath\filename OldDesignName NewTemplateName

[-r] recursively converts through subdirectories of the math you specify.

[-u] upgrades the design of the folders to the same design as $inbox.

Consider the following examples:

• The following command converts all users in the mail subdirectory from their current template to the Mail 8.5 template:

load convert mail/*.nsf * mail85.ntf

• To convert only files from a specific , use the following command:

load convert mail/*.nsf mail8 mail85.ntf

This command skips databases not based on the MAIL8.NTF (Standard R8 Mail Template)

• To convert all mail files to the Mail 8.5 template and convert the design of folders, use the following command:

load convert -u mail/*.nsf * mail85.ntf

• To create a list of mail files by reading people’s mail files from the Domino directory, use the following command:

load convert -l <filename>

• After this list is created, you can edit the list to include only the people you want to upgrade to a particular template. To read the list of databases from the text file and convert only those listed, use the following command:

load convert -u -f <filename> * mail85.ntf

[-f] reads the list of databases to process from a text file

Important: Use extreme caution when using a wildcard character (*). A simple mistake such as forgetting to specify the mail directory will replace all your databases with the Domino Web Access template.

Tip: To easily view switches, descriptions, and some examples for the load convert utility, type load convert -? in the Domino Console.

Backup Strategy

Enabling DAOS servers will have benefits related to disk space savings and reduced I/O demands. However, it will also introduce another level of complexity to your backup / restore planning.

Key elements are already described in the DAOS pages.

Beside the technical changes in how to do back up and how to restore, Domino administrators will have to raise the question if the pre8.5 backup strategy will still be good to go with.

DAOS Strategy

Files in the DAOS repository will never be changed – so it is enough to perform a full backup and from this point onwards taking incremental backups only. This sounds like great potential for saving the number of tapes you need for your backup. Keep in mind that the main purpose for doing a backup is the capability to be able to restore the data in case of a problem. If you have to rebuild your server from scratch, (for example, because of a HW failure), you will find yourself waiting for incremental restores to be applied.

Although Domino itself can be up and running during that time, users wont be able to access all of their file attachments as they have not been fully restored yet.

A definite guide for most efficient backup / restore in your environment cant be provided because the variety of backup software on the market is too large. Also the backup/restore throughput highly influences the optimal configuration.

With our exploration we found taking a full backup of the DAOS repository once per month and taking incremental backups daily provides a good balance for typical customers.

Still we highly recommend to perform backup and restore tests for your environment in order to verify and validate all aspects of your backup / restore strategy.

NSF Strategy

Due to the fact that introducing DAOS (plus other new features) will significantly reduce the size of your NSF files, the sizing of your backup environment needs to be reviewed.

Taking NSF backups will most likely be completed in 30% (or less) of the time it took before. If you are running a shared backup environment (for example, TSM), please review your resource allocation to re-assign tape drive assignments that you might not need any longer.

Transaction Log Backup

Transaction Logs (in archive mode) need to be backed up. The introduction of Domino 8.5 wont change the existing backup configuration.

Unattended Domino Server Installation

When installing Lotus Domino on a larger number of servers, you may want to use an unattended setup routine to speed your deployment. This section provides the details you need to know to install Domino servers automatically on Windows operating systems.

Create a new folder (for example, C:\Install85 ) on the target server, and copy the following files to this new folder

• Domino Installation Package as it was downloaded from passport advantage without extracting it. (C1SP7EN.exe, is Lotus Domino 8.5 Engl. Windows 32 Bit).
• Response file
• Any additional fix pack that you want to apply (For example, CZ344EN.exe, is Lotus Domino 8.5 Fixpack1).

For an unattended Domino Server installation, use the following command line syntax:

<DOMINOPACKAGE> -d -s -a -silent -options "<FULL-PATH-TO-RESPONSE-FILE>"

Example:

C:\Install85\C1SP7EN.exe -d -s -a -silent -options "C:\Install85\Response.txt"

To automatically apply a fixpack, use the following command line syntax:

<DOMINOFIXPACK> -NOUSER

Example:

C:\Install85\CZ344EN.exe -NOUSER

When using the example path names above, a batch file to install Domino and apply a fixpack at the same time could look like this:

@Echo off

ECHO Installation of Domino server code is running in silent mode - This may take some minutes!

START /B /WAIT C:\Install85\C1SP7EN.exe -d -s -a -silent -options „C:\Install85\Response.txt"

ECHO Apply the fixpack

START /B /WAIT C:\Install85\CZ344EN.exe -NOUSER

Sample Response File

REM Attention - see IBM Technote http://www-01.ibm.com/support/docview.wss?uid=swg21299484
REM Note: this file needs to be adjusted by adding the Domino Program + Data drive and path

-G licenseAccepted=true

-P installLocation="<YOUR_DOMINO_PROGRAM_DIRECTORY_HERE"
-W programFiles.isPartitioned=false
-W normalData.installLocationData="<YOUR_DOMINO_DATA_DIRECTORY_HERE"
-W partitionData.numPartitions=0
-W partitionData.installLocationData0="<YOUR_DOMINO_DATA_DIRECTORY_HERE0"
-W ExpressSetupTypePanel.selectedSetupTypeId=CollaborationServer
-W SetupTypePanel.selectedSetupTypeId=EnterpriseServer
-P LicenseFeature.active=true
-P ProgramFilesFeature.active=true
-P bean2090.active=true
-P bean1072.active=true
-P bean2225.active=true
-P bean2247.active=true
-P bean2236.active=true
-P bean2236.active=true
-P DataFilesFeature.active=true
-P bean773.active=true
-P bean3252.active=true
-P bean3301.active=true
-P bean3230.active=true
-P bean3241.active=true
-P bean3271.active=true
-P bean3290.active=true
-P bean3312.active=true
-P DAMOFeature.active=true
-P DECSFeature.active=true
-P DOLSFeature.active=true
-P DWAFeature.active=true
-P SametimeIntegrationFeature.active=true
-P DSyncFeature.active=true
-P DServiceFeature.active=true
-P PerfmonFeature.active=true
-P RMEFeature.active=true
-P HelpFeature.active=true
-P OSIntegFeature.active=true

Console Log Mirroring

One of the new features of Lotus Domino 8.5 is the capability of mirroring the console into a log file.

Console log mirroring causes a new server thread to be created which monitors all messages written to the Console Log file and duplicates these messages into another file. When this new file is filled, the thread closes the mirrored file and creates a new file into which subsequent messages are written. Thus you have all messages not only in log.nsf but also in flat files. You may use them later to parse and send to 3rd party application or search them.

Console log mirroring is not enabled by default.

There are several notes.ini parameters what impact how the Console Log Mirroring is workings.

• Console_Log_Mirror=1 -- Enables the mirroring of console (0 Disables the mirroring (Default))
• Retain_Mirror_Logs=1 -- Prevents deletion of previous mirrors when Domino restarts (0 Deletes all previous console mirror files (Default)).
• Console_Log_Max_Kbytes= -- Sets the maximum size of the Console Log/mirror files (When this limit is reached, a new file is created. The default value is 100,000 (100MB). Maximum size is 1,000,000 (1GB).

When information is written to the console log, it is written also in console mirror files. When console.log reaches a limit, information is written in the beginning of the file. At this stage Domino closes old mirror file, and creates a new one.
The maximum number of console mirror files is 999. The names of the files are the following console1.log, console2.log, console999.log.

You may find additional information on Console Log Mirroring here.
http://publib.boulder.ibm.com/infocenter/domhelp/v8r0/index.jsp?topic=/com.ibm.help.domino.admin85.doc/DOC/H_USING_CONSOLE_LOG_MONITORING_STEPS.html

---

Table of Contents | Previous Page | Next Page