2.7.1 Application management strategy 

Previous | Next

An application management strategy describes an high level approach about how to handle Domino applications within an organization. It is more a concept that can be extended from small implementations to large scale enterprises rather than a fixed set of settings and properties. The application management strategy items described in here are supposed to define the steady state operation for a Lotus Domino environment with a clear goal to improve service quality and process efficiency.

Lets start by clarifying the different services and standards which apply to a production environment.

Infrastructure services

Infrastructure services describe the components which are required to operate a Domino server regardless of its applications. Those services are to be in place before one can start putting production applications on a server.

• Installation of a Domino servers and additional resources:

If required, the infrastructure team is requested to install additional servers according to predefined standards (CPU, RAM, HDDs, and so on).
Defining server categories (small/medium/large) and assigning a specific hardware configuration for each type is helpful here.
• Monitoring of Domino servers:
  Continuous monitoring of running server tasks and server load. Monitoring includes taking care of CPU, memory, and disk utilization.
• Service level reporting:
  Reporting of service levels on a per server basis. If needed, those can be reported for each class of applications.
• Backup and recovery:
  Continuous backup of application data with defined retention periods according to a defined backup plan. Recovery of complete servers or individual applications after a crash or data loss according to defined procedures.
• Virus scanning
  Regular scanning of application data for viruses including definition about how to handle applications which contain harmful code.
• Troubleshooting
  In case of a problem, identify the root cause and take actions to return to steady state operation as soon as possible.
  Analyze the server outages and error messages generated from server crashes.

In order to establish these infrastructure services standards and service levels, a clear description of each element shall be agreed between the involved parties.

Application services

Application services describe the most important services required for operating applications on a Domino server. Those services are typically provided by an application development or application support team in close cooperation with the administration team.

• Maintain an application repository:
  Such a repository is a list of Domino applications with its metadata such as application owner, file name, replicaID, ACL information, operational requirements, expiration date.
  This repository should also contain a classification of the application. Key element is to assign an owner for each application, which can be contacted at a later stage, for example, in case of questions about access rights to the application.
• Rollout of applications:
  Describes the implementation of new Domino applications or additional instances of existing applications according to predefined processes.
  These processes include the collection of metadata in the application repository as well as categorizing applications, quality control, and implementing them into monitoring systems and backup.
• Operation of applications:
  To operate an application, there is more than just providing CPU, memory,or disk capacity. It means operational tasks that are related to the application itself, such as full-text indexing, scheduling agents, or taking care of add-on tasks which have been installed on the server to support this application.
• Troubleshooting:
  Analyze error messages and warnings in case of outages or application errors that lead to partial or complete unavailability of the application.
• Periodic review:
  Regular (for example, every three months) verification of application parameters, data growth, number of users accessing an application and its performance impact. This includes to check if an application that originally was developed as a small-and-simple application requires to be optimized due to its growing number of users.
• Application change management:
  Implementation of changes in production applications, version updates, or functional enhancements should be performed by applying change management rules.
• Access management:
  Change access rights to an application by adding members or roles to the access control list (ACL).
  Note: Maintaining access rights is to be done by the owner of that application who should be allowed to modify the group members, but not be allowed to modify the ACL.
• Creation of usage and billing reports:
  Supply information about the usage of an application to the application owner on a regular basis.
• Rollback and archive:
  On request of the application owner, delete or archive applications from a system.

Not in scope of the application services is the training of users, or the development of individual applications.

Application standards

Application standards in this context lay the foundation for a streamlined operation. By defining and writing down these standards, the operation team and developers will have a common understanding of what rules do apply to the Domino applications in production. Those rules can describe a variety of elements that apply to a production environment. Here are a few examples of what application standards you can define:

• Organizational:
  These type of standards define, for example, who owns an application, how to handle situations where the owner is changing, what is the procedure for creating additional replicas on other servers, and what is to be done for removing an application from the server.
  Do not think about the technical actions, think more about the communication and process point of view.
  One highly recommended organizational standard is to define that each application must have one application owner. Also describe what happens if this is not the case, for example, "if the owner is not identified, the application will be removed from the system by the end of the month."
• Access rights
  What should an ACL look like, are individual user names allowed, or do you only use groups. If so, what naming standard should be applied?
  Is it allowed to create personal or shared views at runtime? What is the maximum access right for typical users, here you do not want to have anyone with Manager or Designer rights.
  What is the maximum access right for developers and application owners in production? Again, in a production environment, only administrators should have design authority.
• Properties:
  Take a look at the properties of a typical Domino application, and define which of those properties are the "standard" in your environment.
  Describe what a setting should look like and why you have decided to enable or disable it. Discuss individual settings with your peers. For example, What should a full text index look like by default? Do you want to index attachments by default? Want to index encrypted fields? What update frequency do you want to use?
  The developers and administrators might have a different understanding because additional functionality often goes along with an increased server impact in one or the other way.
• Design:
  Define a set of common rules that apply to the design of an application. This is not meant to be a complete coding standard definition, but should define corporate design elements, preferred user interface elements, colors, and themes. One example could be to use the IBM OneUI theme for all newly developed applications.

Standards for Domino servers

Similar to the standards for applications, the standards for Domino servers describe how a servers are to be configured in production. Focus is not on exceptions, its more about the settings that will be applied to new servers once they are set up. How to document standards for Domino servers is explained in Chapter 11 1.1 Documentation of the IBM Redbooks wiki "Optimizing Domino Administration".

Application classes

As part of the application hosting strategy, Domino applications should be separated into classes to provide optimal resources by applying different hosting strategies. Categorizing an application into a class is an important factor for optimizing the infrastructure and reducing total cost of ownership (TCO). For that reason, the class of an application cannot be determined by looking at file size, number of documents, or number of users alone.

Small applications that have a small number of documents might be small in size but have a bad application design that can lead to higher resource required than a well-designed large scale Domino applications with thousands of users. By defining classes, application developers and infrastructure teams can work together to optimize the Domino hosting environment for best performance and user satisfaction. For example, a large number of applications that have a low infrastructure impact can be placed on the same server, while heavy applications with special needs can be given the optimal resources by placing them on the servers with just a few other applications or even hosting them on the dedicated servers.

The following categories are examples that can be extended or shortened according to your needs:

• Standard:
  Are most likely based on a (corporate) template and have the following properties:
• No or only small number scheduled background agents.
• Number of views <50 and low frequency of document changes, this being an indication for the view index impact.
• No interface to third party systems, external communication, DLLs, or this sort of special items.
• Fully supports operational standards for backup, monitoring, and support, and is inline with infrastructure standards (for example, Domino version, operating system, and son on)
• Advanced:
  All of the previous class, plus one or more of the following items:
• Are not based on a standard template.
• High amount of users who frequently change documents, this being an indication for the view index impact.
• Unable to work in a cluster.
• Complex
  All of the above plus one or more of the following items:
• Large number of views that are sorted in multiple ways.
• Require add-on programs such as third party libraries that are not part of your standard Domino server.
• Do not comply with the hosting model as they require special handling, for exmaple, a backup concept, or custom replication path.

Once again, the list above is an example, you might want to customize it to your needs. Most important is to describe how to identify which category an application belongs to so that everyone involved in operating an application will have an understanding of these categories. Over time, these categories can be further enhanced so that it is possible to automatically scan an application to identify its category.

Billing framework

Depending on your organization type, you might not need a billing framework because all costs are just covered by your organization. However, certain, especially large- organizations, want to recover the hosting costs from those individuals who are using it, meaning they would like to charge back the cost for each application. Because the size of a database does not indicate how much CPU power it will consume, a key problem here is to find a fair method of identifying the costs of the application. Such a billing framework should therefore, defined on corporate level with measurable criteria that allows all stakeholders to take action for reducing costs.

Describe which building blocks apply to your organization, and if, where, and how you will charge for the Domino applications. This clarification should separate one time costs and recurring costs.

One time costs might apply for:

• Rolling out a new application in a production environment
• Version updates
• Archiving or rolling back an application

The recurring costs might apply for :

• Infrastructure cost, for example, server and network usage, backup, and monitoring
• High availability (clustering)
• Storage consumption
• Troubleshooting and support

An approach to define a billing framework is often done by using the application file size as an indication for its real cost. Although developers and administrators know that this is not the perfect criteria, its often done this way for simplification of the billing process. By using Domino Attachment and Object Service (DAOS) in Domino 8.5, this might become an issue because the real size of the NSF file does not indicate its logical size, nor the logical size indicates how much storage is really used by this application on the servers disk.

This example shows four applications, each 1000 MB in size, but each with a different number of attachments. While the first one does not contain any attachment, the second contains 600 MB attachments, the third 500 MB, and the fourth 400 MB. For each the size displayed to end users is 1 Gbyte.


Where in reality, attachments are not stored within the NSF file in all cases. In our example, the first application is not enabled for DAOS and all other are enabled to participate in single instancing. So each of those applications is using a different share from the DAOS storage. Therefore, charging four times 1 Gbyte would not reflect the reality.

The following approach can be a resolution to this problem:
Lets calculate storage consumption based on the real savings from using DAOS. Savings from single instancing are calculated per server and result in a "DAOS ratio".
This DAOS ratio is calculated by taking the total size of all DAOS objects dividied by (Logical size of all files – Physical size of all files).

For each application, the individual storage consumption is calculated based on the following formula:
physical file size + (Logical Size – Physical Size) multiplied by the DAOS ratio which was calculated above.

As a result, each application will be reported with a file size that reflects the storage used on the server.


The side effects of such a storage reporting system are:

• Owners are not able to verify storage consumption or applications themselves because they cant see the total amount of DAOS storage used by that server.
• Owners can only see the logical file size that is more than he is really using.
• Quota calculation (which may apply to applications) is done based on the logical size.
• The DAOS effect will differ from server to server. If the same application hosted on different servers, it will (most likely) not have the same DAOS savings, and therefore, showing different numbers in a report.
• Large servers with a lot of applications will most likely have a bigger benefit from DAOS than small servers.

Conclusion

When all these standards and procedures are described and are published so that everyone in the organization can access them at any time, a foundation for smooth Domino application operation is set. Describing the background and process details will help to streamline application operation.

Parent topic: 2.0 Application development lifecycle