ShowTable of Contents
IBM Lotus Quickr for WebSphere Portal allows you to back up, archive, and restore places just as you back up, archive, and restore composite applications in IBM WebSphere Portal. This article explains how this this feature works in the Lotus Quickr 8.5 release.
Changes in this release
Although this feature has been part of earlier releases, there have been significant improvements made in the Quickr 8.5 release.
Following are key new additions to the backup-archive-restore feature:
- An improved user interface is available to administrators for managing backup-archive-restore operations on each Quickr server instance. This user interface is part of the Quickr Administration Console, which is a new feature in Quickr 8.5. To access these operations, select Places Administration, located at the bottom of the Lotus Quickr window.
- Starting in Quickr 8.5 for WebSphere Portal, administrators have access to a service called the Backup Scheduling Service, which helps automate the task of taking regular backups on individual places. This service can be used by administrators to specify the set of places that need to be backed up on a regular interval. The service also allows customizing the backup schedule on those places.
Although the product does not ship any UI to work with the service, there is a supported Representational State Transfer (REST) API that can be used to quickly build a custom UI to work with this service. Alternatively, you could use the sample UI published as part of the API documentation in the Lotus Quickr wiki. Note, however, that the sample is provided as-is and hence no support is available from IBM.
Following are the key functional changes in Quickr 8.5:
- Backup-archive-restore operations are available only to server administrators. In previous versions, place owners also had access to these operations.
- There is no known limitation on how big the place should be for a successful backup. Earlier releases had a constraint that places whose size exceeds 2 GB could not be backed up successfully.
The Composite Application Infrastructure (CAI) framework manages the overall process of backup-archive-restore. Each component in the place performs the actual work of backup-archive-restore of the content owned by its instance. In addition to delegating the content-level operations to components, CAI handles the backup-archive-restore of component settings, place settings, membership, roles, layout, etc.
Backup Storage Layout
Figure 1 labels the backup layout on the file system.
Figure 1. Backup layout
Backups are stored on the file system, whereas some metadata about the backup are stored in the database.
The backup location on the file system is configurable. By default backups are stored at <Quickr_Install_Root>/<wp_profile>/PortalServer/backup.
When a backup is created for the first time on the server, a root directory with a name matching the ObjectID of the Virtual Portal is created in the backup directory. Under this root directory, each place backup is represented by a folder with its name matching the ObjectID of the place.
Underneath, there is a unique folder for every backup instance (also known as restore point) of the place. Under this folder, there are three XML files, application-instance.xml, restore-point.xml, and application-favorites.xml. Finally, there is a folder for each component instance, in which the component data is serialized into multiple data files.
Understanding the backup detail
While the contents stored by components are not human readable, some details about the backup can be read directly by looking at the backup directory.
First, the restore-point.xml file contains the metadata on the backup instance, as shown in listing 1.
Listing 1. Example metadata on backup instance
<?xml version="1.0" encoding="UTF-8"?>
<backup:application-info id="1F_18M131M41G1000IIQES9RV1C70" folderID="ibm.portal.
<backup:nls-string xml:lang="en">Meeting Place</backup:nls-string>
<backup:nls-string xml:lang="en" />
As we can see from the XML, it contains the info on who created the backup, the status of the backup, and the place details. The status of the backup can be WORKING, COMPLETE, FAILED, PARTIAL. The status PARTIAL is used to indicate when there is an exception thrown from one or more components while the overall backup still completed. The status will be marked FAILED when CAI itself encounters exceptions when backing up the place.
Next, the place details are captured in application-instance.xml file. This file stores the place layout, pages, roles, members and metadata at the component instance level.
Backup Scheduling Service
Backup Scheduling Service is a new service in Quickr 8.5 and is accessible only to server administrators. The service uses a WebSphere Application Server scheduler to periodically (by default, once a day) check for changes on places queued up for scheduled backup and kicks off backup on those places that have changes.
Places must be added to the queue in order to automate their backup using this service. There is a published REST API for using this service, and there is also a sample portlet provided to assist is using this service. However, there is no supported UI for this service in this release. More details are available at the API documentation wiki page for Backup Scheduling Service REST API
and Sample Portlet for Backup Scheduling Service
The Quickr Administration Console, labeled as "Places Administration," provides the user interface for managing backup-related operations. Using this UI, the administrator can do the following (see figure 2):
Figure 2. Back Up Versions window
- Start the backup for a place
- View the status of existing backups on a place
- Restore or delete a backup
- View all archives available on the server and their associated backups
Architectural changes in this release
In previous releases, components created a .zip file for holding the backup of their contents. In Quickr 8.5, this has been replaced with a file stream that uses multiple files when storing large content.
Quickr 8.5 is based on WebSphere Portal 6.1, which contains extensive changes in the CAI code compared to that of WebSphere Portal 188.8.131.52 used in earlier releases of Lotus Quickr. For backups the biggest difference is that the information about existing backups is no longer read from the filesystem but from the database. Consequently it is no longer sufficient to copy a backup from another system into the filesystem.
Instead, to import backups into the database, you must also run an XMLAccess script. Details about this change can be found in the topic Synching the interface to update the list of places
To plan your backup process and structure, review the product documentation topic, Backup and recovery considerations
For steps and menu selections, refer to the product documentation topic, Backing up and archiving places
If you experience problems with backup-archive-or restore, gather answers for the troubleshooting questions and steps, then set trace if diagnostic data needs to be collected.
Questions and steps
For backup / archive issues
- Platform, operating system, service pack
- Product fix pack levels
- Logs (System Out.log, System Err.log)
- What is the status of the backup when viewed in Places Administration, the administration UI?
- Does the place remain as "locked"?
- What is the backup location? Is it accessible from the node where the service is running?
For restore issues
- Place information: place owner, place type (private/public/self-join)
- Component details: What are the components in the place? Are there any custom portlets/components in the place? Are any of the components from a previous release?
- Who initiated the backup/archive operation? Does that user belong to a role on the place?
- Was the place migrated from an earlier release?
- What template was used to create the place?
- Was the template migrated from an earlier release?
- Can we try to delete the failed backup and try the backup again?
For Backup Scheduling Service Issues
- Review the application-instance.xml file from the backup folder
- Was this backup migrated from an earlier release?
- Who is doing the restore?
- Can we register the backups once again and then try the restore?
- Are there any places that have changed since the last time the service ran?
- Current configuration of the service (interval, duration, # of backups)
- Selection criteria (List of place IDs, template IDs in the queue)
- Place sensor task configuration
- Is this a cluster? If so, which node is running the service?
- Does deleting the task info record from table <RELEASE_SCHEMA_NAME>.QUICKR_SCDTASK and restarting the node resolve the issue?
The IBM Support Technote, "Collecting Data: Lotus Quickr for WebSphere Portal 8.5,"
provides information about generating and collecting diagnostic data. For backup-archive-restore, you can use the traces below.
1) If the backups are not showing up in Places Administration, set this trace:
Get the Firebug console output when viewing the backups in the Places Administration
2) If the backup-archive-restore operation results in an error and/or the place remains locked for a long time, set these traces:
3) If places added to scheduled backups are not getting backed up, and for any other issues related to Backup Scheduling Service, set this trace:
4) For any other failures, enable the following component-level tracing after looking at the initial exception stack, to pinpoint where the component is failing.
Best practices for problem diagnosis
- Make sure the backup location is accessible and has sufficient disk space
- The size of the place displayed in the Administration Console should closely match the size of the backup folder for that place at the time the backup was created. If not, something is wrong.
- When in doubt, check the server logs for exceptions, even when the status of the backup says OK
- When a place that has backups gets deleted, those backups will be displayed as archives
- Check the database tables when the logs are not sufficient to form any conclusion
Considerations for problem diagnosis
In backup-restore, the important aspect to keep in mind is NOT to lose whatever data is available, even if it is not the entire set that you would have liked to preserve. Hence, it is always safe to make a copy of the place backup folder before doing any diagnosis. This way, you do not accidentally lose the backup as part of the investigation.
Other details, such as the type of content that was stored in the place prior to backup-restore operations, would be useful in charting the course of the investigation. For instance:
- Did the document use any document template / custom document type?
- Was the place created using any custom template?
- Were there any custom roles that were created in the place?
- Are there any missing users in the LDAP who the place is listing as a member/owner?
When a restore is performed, the existing place is deleted first. In case something goes wrong during the actual restore process, the place is turned into an archive. So if the place is not listed anymore, you should check the archive section.
Common errors and workarounds
Backups were copied from one server to another but were not registered. Hence they will not be listed in Places Administration (Administration Console UI).
Run the backup registration xmlaccess script as described in the product documentation Moving a place between Lotus Quickr servers
Places were scheduled for backup, but the sensor task runs after the backup scheduling service runs. Hence the place's timestamp did not get updated, and the place was not backed up automatically.
Always schedule the sensor task to run ahead of Backup Scheduling Service. For example, if the backup service is scheduled daily at 11:30 p.m., then schedule the sensor task to run at around 10:30 p.m. If there are a large number of places, the sensor task might run for a longer time. You can trace com.ibm.wps.ai.scheduler.*=all
to check the task execution.
Applications (Quickr Customization, Scheduled_Backup_Service) are either missing the security role mapping or they are mapped to non-admin users.
Map the security role to the server administrators group by using the WebSphere Application Server Administration Console.
Sometimes a restore fails because the re-creation of the data clashes with existing data. In this case duplicate key exceptions are listed in the logs.
Remove the existing resources by hand. If possible, it should be investigated what failed earlier to get into this state, so that the underlying problem can be fixed.
Can you do offline backups or back up to a different disk or LPAR (in the case of hardware failure)? In other words, you want to choose a different location or machine for storing the backup.
Yes, as long as the location can be reached through the file system from the Quickr server, it's possible. For example, you could select a network directory to store the backups and even use that location for multiple Quickr instances. To customize the location, follow the "To perform backups on a cluster" steps in the topic Backing up and archiving places
Some organizations have requirements to "lock" backups, to prevent a backup from being overwritten by a newer backup. Does Lotus Quickr have this function?
No, there is now way to lock a specific backup. However, you can manually copy the backup from file system to another storage place, to make sure it is not overwritten.
If a place is deleted or moved, are all the backups automatically also removed from the system?
No. If you delete a place that has backups, the Quickr server automatically treats them as an archive. The administrator can see the deleted place in the archive area of Places Administration, but the deleted place is no longer visible to users.
About the author
is an Advisory Software Engineer who has worked on Lotus Quickr since its initial release. He is currently a member of the Quickr for WebSphere Portal team and, during Quickr 8.5 development, he led the development for Backup-Archive-Restore and Templates features.