ShowTable of Contents
Overview
Migrating Web content from heterogeneous systems is always a challenge, given the disparate content structures and components required to present the data.
This article describes a process to migrate the core content from the Vignette Content Management System to IBM® Web Content ManagerTM (WCM). A Web Content Migration Tool (WCMT) Web application has been created to facilitate the entire process, which can be executed through eight simple steps. The output of the process is the WCM Libraries containing the WCM artifacts converted from the Vignette content.
Although the majority of the Vignette content is automatically migrated, some manual steps are still required; for example, creating authoring templates and fixing the legacy links in the content.
We explore tips and workarounds for handling specific Vignette features and introduce a framework that generalizes any content migration so that the content source is not just limited to Vignette. Figure 1 illustrates the system overview.
Figure 1. Web content migration overview
The WCMT application parses the exported Vignette XML files and generates the RSS Feeds file to be imported to the IBM WebSphere® Portal / WCM Server through the Web Content Integration (WCI) feature of the Portal Administrator Console.
The WCMT properties file provides the environment information, and the WCM API is used to complement the functionality not covered by the WCI; for example, creating content links and moving library components into the folders.
For simplicity, the current tool supports only the full migration with one site at a time and a full content import, as opposed to the delta import that affects only the changed items.
Vignette-to-WCM feature mapping
Since the Vignette Content Management System uses different terminologies for its implementation, a mapping is required to ensure an equivalent functionality is implemented in WCM. Table 1 provides a feature/component mapping between Vignette and WCM.
Table 1. Feature/component mapping
Vignette feature | WCM feature | Migration method | Comments |
Site | Root Site Area | Automated via WCI | A WCM content library can contain multiple sites. |
Channel | Site Area | Automated via WCI | Vignette uses this as the user’s navigation hierarchy. |
Content Type Definition (CTD) | Authoring Template | Manual | This is for content item type only, stored in the design library, and shared across all content libraries. Presentation page and components are excluded. |
Content Instance | Content Item | Automated via WCI | Stored in a content library |
Static files | 1. Content items with embedded document
2. Library Image or File Component
| Automated via WCI | Images are always stored as library image components, whereas documents (.doc, .xls, .pdf) can be configured and stored as either content items or library image components. |
Workflow | Workflow | Manual | Some updates to the existing workflow process may be required. This is stored in the design library and shared across all content libraries. |
Classification | Categories | Automated via WCI | Can be configured to be stored in a local content library or design library |
Figure 2 shows a clear picture of the WCM Library layout.
Figure 2. WCM Library layout
Vignette CTD-to-WCM authoring template mapping
All the mapping information between the newly created authoring templates and their corresponding CTD is described in the WCMT.properties file. For example, the following line indicates that Vignette-CTD1 is mapped to the WCM authoring template, WCM-AT1, and so on so forth:
AuthoringTemplates=Vignette-CTD1#WCM-AT1,Vignette-CTD2#WCM-AT2
The mapping information between the Vignette CTD attributes and the WCM Content elements is also defined in the WCMT.properties. For example, the following line indicates that attribute1 of the Vignette-CTD1 is mapped to element1 of the WCM-AT1:
Vignette-CTD1@attribute1=element1
Table 2 shows the element syntax and types that are currently supported by the WCMT application.
Table 2. Element syntax and types
Syntax | Comments |
element-name | Mapped directly to the system-defined or custom element. The following system-defined element names are supported:
Name, DisplayTitle, Description, Keywords, PublishDate, ExpiryDate, Author, Owner.
|
type@element-name | type defines the element type, for example, Rich Text. The following element types are supported:
- rt – rich text
- date
- number
- cmpntRef – component reference
- extLink – external link
- intlink – internal link
- anchorLink – anchor link
- optionselection
|
Content import prerequisites
Prior to performing the automated content import process, the following fundamental WCM objects must be in place:
- Create Design Web content library. Two types of WCM libraries are defined: The Design library that hosts design assets such as templates and the content library, generally one library per site / virtual portal. Because content requires certain design assets to exist as a prerequisite, the design library defined in the WCMT.properties file must exist first. For WCMT to work, “wcmadmin” must be added to the Administrator role of the library.
- Create the Web content library to map the Vignette site. This library, defined in the WCMT.properties file, stores content instances, static files, and classifications for the Vignette site, and will be the target of the subsequent activity of performing the actual content import. For the WCMT Tool to work, the “wcmadmin” must be added to the “Administrator” role of the library.
- Create workflow objects. All content must have an associated workflow and stage. All workflows and stages must be manually created in the design library.
- Create authoring templates. All content requires an authoring template, which defines the fields available to the content item, as well as some other aspects such as help text and default values. The authoring template in WCM is akin to the CTD in Vignette. All authoring templates must be created in the design library.
Import content using WCI
WCI is a portal feature that relies on a specifically formatted RSS feed to import content into WCM. The output of the WCMT application is the RSS feed ready to be imported to the WCM.
WCMT is not a 100%-complete migration solution. Some analysis is required to be successful, and there are both prerequisite steps and post-import steps that must occur. However, it does serve to save significant time from the actual moving of content data from the legacy content management system to WCM.
Setup
Table 3 shows the resources required to deploy and execute WCMT in the WebSphere Portal environment.
Table 3. Resources required
Name | Installed Location (example) | Comments |
WCMT_EAR.ear | C:\IBM\WebSphere\wp_profile\installedApps\portal\WCMT_EAR.ear | Install via WAS Admin Console |
WCMT.properties | C:\IBM\WebSphere\PortalServer\shared\app\WCMT.properties
NOTE: Each site has its own WCMT.properties. Ensure you have the correct WCMT.properties in place and then restart the WCMT application before building each site. For example:
library=WCMT_Site_1
designLibrary=WCMT_Design
| Install by copying the file |
Vignette Exported XML files | C:\wcmtFiles, can be specified as a property – inputFileDirectory - in the WCMT.properties | Install by copying the files |
WCM Resources | WCM Content Library; e.g., Site1 – in WCM repository
WCM Design Library; e.g., Design – in WCM Repository
WCM Workflow; e.g., General Workflow – in Design library
WCM Workflow Stages; e.g., General Draft, General Ready for Approval, General Publish, General Expire, and General Reject – in Design library
WCM Workflow Actions; e.g., Publish – in Design Common library
WCM Authoring Templates – as described previously – in Design library
WCM Temp Library and content – Temp/Trash/ReferenceCollection – required only when using the “Delete Site area” command in the tool.
| Create by using the WCM Authoring Portlet |
WCMConsumerPlugin. properties | C:\IBM\WebSphere\wp_profile\PortalServer\wcm\shared\app\config\
WCMConsumerPlugin.properties
max.filesize.mb=50
wcm.servlet.path=/wps/wcm/myconnect
disable.img.proc=true
| Manually modify the properties to support maximum image size |
JVM Settings | Initial heap size: 1024 MB
Maximum heap size: 6144 MB
-Xjit:exclude={oracle/jdbc/driver/T4CTTIfun.receive()V} => a workaround to fix the Oracle XA Exception
| Manually modify via WAS Admin Console |
Portal Virtual Resources Setting | | Manually modify via the Portal Admin Console |
Since the WCMT program is quite resource intensive, it is highly recommended that full resources be allocated to run it. During the run, make sure no maintenance programs (for example, DB Backup) are running in the background; otherwise, performance will be affected.
Generate RSS
The WCMT Web application supports the following features:
- Format exported Vignette XML file with new line. This adds new line character to each tag and makes debugging much easier. Note that WCMT can only process XML files that have been formatted with new line characters.
- Generate RSS Feed XML file. The tool can generate RSS Feed XML file from various Vignette object types, for example, Channel, Static File, and Content Instance. The RSS Feed XML files are later consumed by the WCI of WebSphere Portal to create WCM objects.
- Generate unresolved link reports and text files. This identifies legacy links and missing static files and allows you to take appropriate action to fix the issues.
- Post-feed action to move library image components into correct folders. All images are initially created as library image components in the component root folder after the Static Files feed XML is consumed. This action moves the image components into the folders.
- Post-feed action to create content links. Content links are used to represent the content instance assignment to the channel. This is specific to Vignette.
- Clean up Site Areas. Site Areas in the content root can be selected for removal. It is recommended to use “Delete Library” from the Portal Admin Console, unless the command fails due to cross-link references between libraries. It has been recommended that, in the future, the WCMT be extended to include the feature of dynamically removing the cross references, to facilitate the “Delete Library” command.
The process for the content migration requires two passes due to the circular dependency of the links among multiple sites (see figure 3) and the inability to download all the static files from Vignette in one shot.
Figure 3. Link circular dependency
The first pass is to generate unresolved or missing-link reports from each site and then merge them. For example, if Site 2 contains a link to Site 1, it must be determined whether the link must is already in the Static Files of the Site 1. If it is not in the original download of the Static Files, then it must be added as a missing static file to the Site 1 missing list, so that the Administrator can include that in the second download.
Since there is circular dependency, no matter which site you start with, you will end up with broken links that require manual fixing. To minimize the manual operation, we choose the site that has most references from other sites to build.
The general procedure to complete a site migration is as follows:
(1) Format Vignette’s exported file with new line characters.
(2) Generate the feed XML file for the Static File Images.
(3) Generate the master feed XML file for Static File Documents, Content Instances, and Channels. Unresolved link reports are also generated as part of the result.
(4) Run Steps 2 & 3 for each site, analyze the missing links, and merge them until all the static files from Vignette have been downloaded.
(5) Consume the feed XML file from Step 2.
(6) Run the post-feed action to move library image/file components into the folders.
(7) Consume the master feed XML file.
(8) Run the post-feed action to create content links in the channels.
Here are the detailed steps to generate the feed XML files for the Sample site:
- Ensure that WCMT.properties is in place and the WCMT Tool application is restarted.
- Copy the exported Vignette XML file to the C:\wcmtFiles directory. Note that the Vignette binary static files must be copied to the C:\wcmtFiles\StaticFile directory.
- Bring up the WCMT Tool, using http://localhost::10039/wcmt/start, select “Add New Line to File” in the Command field, and enter “Sample.xml” in the Input File field (see figure 4).
Figure 4. Add New Line to File command
NOTE: The generated OUT_Sample.xml output file will be used through the rest of the process.
4. Now select “Generate Feed File (Channel/Site area)” for the command, and OUT_Sample.xml for the input file. Click Submit to generate the feed file, Feed_Channel_OUT_Sample.xml, for the Channels.
5. Select the “Generate Feed File (Static Files – Image)” command and the OUT_Sample.xml input file. Click Submit to generate the feed file, Feed_StaticFile_Component_OUT_Sample.xml, for the library components. Since the documentComponent property is set to false, documents such as PDF or DOC will be generated as content items.
6. Generate the missing static files report by running the command “Generate Feed File (Content/Document/Channel)” (see figure 5).
Figure 5. Generate Feed File (Content/Document/Channel) command
a) Check the Message_WCMT_Site_1_UniqueMissingLinks.txt in the wcmtFiles directory and ensure each line starts with /vcm_content/../ (see figure 6). Modify any lines that start with “http:” or “https”, so that they all fall into the same /vcm_content format and remove links to other sites.
Figure 6. Message_WCMT_Site_1_UniqueMissingLinks.txt
b) Also, check the StaticFile directory; you should see that all the VGNCONTENTS binary files have been renamed to the target names. Rename the StaticFile directory to StaticFile_main.
c) Repeat Step 1 through Step 5 for other sites so that all the cross-site links to the WCM_Site_1 site are merged into one file named MissingItems.txt. If there are no missing links or only a few of them that can be fixed manually, skip Step 7 and go directly to Step 8 to start importing the feed files.
7. To ensure Step 1 is executed for the WCM_Site_1 site, create the StaticFile directory under C:\wcmtFiles and copy the missing binary static files to the StaticFile directory. Then copy the new Vignette XML file (Sample2.xml) that defines the missing static files to C:\wcmtFiles and perform the “Add New Line to File” command to format the XML file with new line characters.
a) Use the newly formatted XML file as the input for the next command, “Filter Missing Items”, as shown in figure 7, after ensuring the MissingItems.txt is in the C:\wcmtFiles.
Figure 7. Filter Missing Items command
The purpose of this command is to filter the static files by the entries in the MissingItems.txt file. The result is the FilterMissingItems_OUT_Sample2.xml file that contains the missing Static File XML entries, which can be incorporated into the original OUT_Sample.xml.
Each entry contains a reference to a VGNCONTENTS_x[n] static file, where x is specified in the filterChar property (see figure 8). This process lets you incrementally add static files to the main set. Make sure you change the filterChar property to a different value, for example, y, z, etc., to avoid the name space collision.
Figure 8. VGNCONTENTS files
b) Copy all the VGNCONTENTS_x[n] to the StaticFile_main directory and rename the StaticFile_main back to StaticFile. Copy all the “<importStaticFile>…</importStaticFile>” entries from the FilterMissingItems_OUT_Sample2.xml file, paste them into the OUT_Sample.xml file, and rename it to OUT_update_Sample.xml.
8. Repeat Step 4 as follows: Select the “Generate Feed File (Static Files – Image)” command and the input file, OUT_update_Sample.xml. Click Submit to create the feed file for the library components.
9. Consume the Feed_StaticFile_Component_OUT_update_Sample.xml file, after ensuring that the feed file and the C:\wcmtFiles\StaticFile” directory are copied to the URL addressable target directory specified as the RemoteURLPath property in the WCMT.properties.
a) Log in to Portal and navigate to Administration --- Feed Configuration. Make sure the credential slot is created and that the Feed URL field points to the location of the copied feed file (see figure 9). Click Save, to complete the Feed configuration.
Figure 9. TestFeed window
b) Click Consume to start consuming the feed that will import library components to the root of the library (see figure 10). Check the SystemOut.log for the completion status.
Figure 10. Web Content Integrator Feeds window
After this is done, you should see library components created in the library (see figure 11).
Figure 11. Library components displayed
10. Run the post-feed action to create the component folders and then move the components into the folders.
11. Select the “Generated Feed File (Content/Document/Channel)" command and the input file, OUT_update_Sample.xml (see figure 12). Click Submit to generate the master feed file.
Figure 12. Generated Feed File (Content/Document/Channel) command
This command also generates the reports shown in table 4.
Table 4. Command reports
Name | Description |
Message_WCMT_Site1_Missing Link Error.html | A list of static file links that can’t be resolved. |
Message_ WCMT_Site1_UniqueMissingLinks.txt | A unique list of static file links that can’t be resolved. |
Message_ WCMT_Site1_Link Error.html | A list of unresolved (relative and absolute) URL and content links, some of which require user’s attention or manual fixing. |
Message_ WCMT_Site1_ Link Not Handled.html | A list of unhandled absolute URL for user’s review |
Message_ WCMT_Site1_ UniqueLinksNotHandled.txt | A unique list of unhandled absolute URL for user’s review. |
12. Now, consume the master feed XML file, Feed_Content_Document_Channel_OUT_update_Sample.xml, after ensuring that the feed file is copied to the URL addressable target directory specified as the RemoteURLPath property in WCMT.properties.
a) Update the Feed Configuration to reflect the correct XML file as before.
b) Click the Consume button to start the content import, and then check the SystemOut.log for the completion status.
13. Finally, run the post-feed action to create content links in the channel site areas (see figure 13).
Figure 13. Post Content Instance Feed Action command
Handling embedded links
Links embedded in the rich text element are processed during the content feed generation. Table 5 lists the link types in the content that are handled.
Table 5. Link types handled
Link types |
<img src=”[url]”../> |
<a href=”[url]” > <a> |
window.open(‘[url]’… |
window.location=’[url]’ |
Note that the link types are specified in the WCMT.properties file and can be extended:
embeddedLinkPattern=src\="@",href\="@",.open('@',.location\='@'
Generalization framework
The WCMT application contains a built-in framework to allow customization, providing the opportunity for migrating content from any content management systems to WCM. Figure 14 shows the class diagram of the application.
Figure 14. WCMT class diagram
The ChannelPlugin, StaticFilePlugin, and ContentInstancePlugin classes are replaceable and can be specified in the WCMT.properties file:
channelPluginClass=com.ibm.issc.wci.tool.ChannelPlugin
staticFilelPluginClass=com.ibm.issc.wci.tool.StaticFilePlugin
contentPluginClass=com.ibm.issc.wci.tool.ContentPlugin
All the plugin classes implement the WCMTPlugin interface and the getList method, which returns a hierarchical list of WCINode objects. The content plugin class implements additional two methods, processLinkURL and processImageLinkURL, to support embedded-link handling.
For this to work, all the customized classes must be packaged as a .jar file and placed in the C:\IBM\WebSphere\PortalServer\shared\app directory. For details, contact the author at fchen@us.ibm.com or ISSC (IBM Software Services for Collaboration) for assistance.
Conclusion
The Web Content Migration Tool has been introduced to assist users in migrating content from the Vignette CMS to WCM by use of the WCI feature of the WebSphere Portal Server. The RSS Feed XML files of the Vignette content can be easily generated through the UI, validated, and then imported to WCM.
A generalization framework is also built into the application, allowing further extension to various content sources. Contact ISSC for more detailed information or, if you need to use the application, contact fchen@us.ibm.com.
Tell us what you think
Please visit this link to take a one-question survey about this article:
Resources
About the author
Fetchi Chen is a solutions-oriented Senior Management Consultant in the IBM Software Services for Collaboration group. He is a certified Consulting IT Specialist experienced in application development on various hardware platforms and operating systems. He has over twenty years of software development experience designing solutions for numerous industries, recently specializing in WebSphere WCM/Portal-related solutions.