This article describes about the settings recommended for the successful migration of large files from IBM Portal Document Manager library to Lotus Web Content Management using the IBM WebSphere Portal Document Manager migration tool. Ensure that the following settings are configured on your system before proceeding with the migration of the large files.
In Lotus, Web Content Management, resource.maxUploadSize
is a setting () that specifies the size of the largest file that can be imported, if not specified then by default this value is 16MB. The value can be set in the WCMConfigService.properties file which is located in:
In order to change this value and apply related settings for large file handling, follow the steps below :
Edit WCMConfigService.properties to change the value for 'resource.maxUploadSize' to desired value which is specified in MB. (Please follow the steps in the link)
1. Update the WCMConfigService.properties file
a. Ensure that the resource.maxUploadSize property is uncommented, and specify a value in megabytes corresponding to the size of the largest file that you want to allow to be imported. For example, if you do not want to allow files larger than 34 MB to be imported, you would specify this. resource.maxUploadSize=34. It is recommended that this value not exceed 100 MB.
b. Ensure that the resourceserver.maxCacheObjectSize property is uncommented, and specify a value of 300 KB or less.
c. If it doesn't already exist, add the transaction.sync.remove property to the file, and specify a value of true. For example:
transaction.sync.remove = true
d. Save your changes.
2. Add the protocol_http_large_data_inbound_buffer property
Log in to the administrative console for IBM WebSphere® Application Server
WebSphere Application Server 6.1: Servers -> Application Servers -> -> Server Infrastructure -> Administration -> Custom Properties.
WebSphere Application Server 7.0: Servers > Server Types > WebSphere application servers > portal_server > Server infrastructure > Administration > Custom Properties.
a. Click New.
b. Enter protocol_http_large_data_inbound_buffer in the Name field.
c. In the Value field, specify the maximum file size in bytes. This value should correspond to the value you set for the resource.maxUploadSize property in the WCMConfigService.properties file.
Set the value for the largest file size eg. 476mb - 488681472 bytes
Note that the protocol_http_large_data_inbound_buffer property uses bytes. So if you specified a value of 34 MB for the resource.maxUploadSize property, you would specify a value of 35651584 bytes for the protocol_http_large_data_inbound_buffer property.
3. Verify that the fullyMaterializeLobData property value is false
This property can be set in these files :
WebSphere Application Server 6.1: Resource > JDBC > Data sources > datasource_name > Custom properties.
WebSphere Application Server 7.0: Resource > JDBC > Data sources > datasource_name > Custom properties.
a. Ensure that the fullyMaterializeLobData property is set to false. Create a new property if not already there.
The effect of fullyMaterializeLobData depends on whether the database server supports progressive locators:
See below for more information.
LOBs in JDBC applications with the IBM DB2 Driver for JDBC and SQLJ
http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=/com.ibm.db2.udb.apdv.java.doc/doc/cjvjdlob.htm">See for DB2 9
See for DB2 8
4. Increase the maximum number of database connections allowed for the application server
This property can be set in these files :
WebSphere Application Server 6.1: Resource > JDBC > Data sources > datasource_name > Connection pool properties.
WebSphere Application Server 7.0: Resource > JDBC > Data sources > datasource_name > Connection pool properties.
a. Increase the value of the Maximum connections field to a value greater than the default 50 connections.
Set to 120 for large file upload for the file size from 200 MB – 500 MB
5.Transaction Timeout setting
If you are working with files larger than 100 MB, increase the Web containers transaction timeout setting:
WebSphere Application Server 6.1: Servers > Application Servers > portal_server > Container Services > Transaction service.
WebSphere Application Server 7.0: Servers > Server Types > WebSphere application servers > portal_server > Container Services > Transaction service.
a. Increase the value of the Total transaction lifetime timeout
and Maximum transaction timeout
setting from the default setting of 120 seconds.
Set them to 1200 seconds (20 minutes)
Increase the maximum number of threads allowed in the thread pool used by the Web container:
WebSphere Application Server 6.1: Servers > Application Servers > portal_server > Thread pools > WebContainer.
WebSphere Application Server 7.0: Servers > Server Types > WebSphere application servers > portal_server > Thread pools > WebContainer.
a. Set the value of the Maximum Size field to 100 threads.
7. Change LTPA session Timeout in Productivity Components
Default value is set to 2hrs(120). Change it to 1440 (24 Hours).
WebSphere Application Server 6.1: Security > Secure administration, applications and infrastructure>Authentication mechanisms and expiration> [(Default value) Key set group(=NodeLTPAKeySetGroup)]>Timeout value for forwarded credentials between servers > set to 1440.
8. Click Save
to save your configuration changes.
9. Restart Portal
for the settings to take effect.
Other Recommended Settings
If you are migrating large files, increase the JVM heap value to avoid getting an 'OutOfMemoryException'.
The JVM Heap can be set to 1500 or 2000 depending on the amount of RAM available on your system.
Login to admin console, select Application servers > WebSphere Portal > Process Definition > Java Virtual Machine, Change Maximum Heap Size field.
XA Transactions Errors
It is recommended to disable XA transactions which sometimes run at the background. They may interfere with the migration of the large data/files. XA transactions can be disabled by following the steps below.
a. stop the server
b. delete /tranlog folder
c. start server
JCR text search
It is recommended you disable JCR text search indexing on your WebSphere Portal 6.1 server before migration. Doing this will dramatically reduce the load on the database during migration and syndication. To do this, edit was_profile_root/PortalServer/jcr/lib/com/ibm/icm/icm.properties and set "jcr.textsearch.enabled" to false.
After the file is updated, restart your server for the changes to take effect. You must re enable this after migration. It may take some time to rebuild the indexes when you re-enable JCR text search indexing after migration.
Large Document Preview
To preview large documents, make sure that the Maximum preview size limit is set
1. Navigate to Portal Administration -> Portlet Management -> Portlets -> Document Manager -> Configure Portlet.
2. Add the portlet parameter dm.view.document.previewSize.max, set the value.
Note- If using DB2, make sure that fullyMaterializeLobData property is set to false (not that the default is true) in the JDBC driver to allow streaming of large files to avoid an out-of-memory