Performance best practices

 This page collects some best practices for Portlet Factory related to performance and memory. This is not a comprehensive list, and most of the best practices for the underlying platforms – WebSphere Portal, J2EE and WebSphere Application Server, and Java – are also applicable for Portlet Factory applications.

Contents

• Avoid retrieving large data sets, particularly in session variables.
• Use caching across users when feasible.
• Avoid deploying a large number of Portlet Factory WAR files.
• Use the latest version and fixpack of Portlet Factory.
• Use Portlet Factory's built-in logging tools to look at performance-related information.
• Use automated load testing tools to validate the performance and capacity of your application.
• See the WebSphere Portal and WebSphere Application Server tuning guides for recommendations to optimize server performance.
• Be aware of the impact of Portlet Factory profiling features.
• When using profiling for Edit (Personalize) support in Portal, use the "Execution Time" setting for profile entries.
• Utilize the capabilities of external systems when possible.

Avoid retrieving large data sets, particularly in session variables.

Large data sets are the most common cause of memory capacity issues. Here are some suggestions for how to keep data sets smaller.

• When retrieving data, try to retrieve a small amount for viewing at a time. Depending on your data source, you can often reduce size with a more restricted query (to reduce the number of rows and/or number of columns retrieved). With SQL builders, you can use a selective query to do this. With Domino, you might consider using or creating more selective views in Domino, to reduce the row and column count.
  If you can't restrict the data set size from the data source (for example, if you only need to display a few columns while your data source returns a lot of columns), you can create a service provider that removes unneeded data, so that the data used by your consumer model (and potentially stored in session) is smaller.
• If you have a large set of categorized data, consider providing an interface that initially shows just a list of the categories, with links to retrieve data for a single category at a time. This technique uses much less memory than retrieving the entire data set and displaying it with Category Row builder, for example.
• Caching of results can be another technique for sharing data across users and reducing session memory use. For example, if you enable caching in the Service Operation builder, all users (for the same profile) will share the same result object. This sharing will occur even though the result variable may be session-scoped in the consumer model - each user session will have a reference to the same Java object.
• Use paging to fetch data in subsets. There are a number of builders in Portlet Factory that support a UI for paging through tabular data sets. For example, there are paging options in the View & Form builder and the Data Page builder, and the Paging Assistant builder provides some of the underlying support. These paging features can use a "DataRetriever" interface that can be used to automatically fetch data in chunks from an external source. Some data access builders such as SQL Call and Domino Data Access can provide support for a paging DataRetriever interface. Other data access builders such as SAP Function Call do not have any built-in support for paged data access. There are two samples that show some custom support for paging data: IBM - Paged web service provider and IBM - Creating a custom data paging provider. When you are using Data Page builder to display paged data, be sure to select the top-level variable as the data to display, not an XML element within the variable.
• For large variables, if they can be marked as "Shared (read-only)", they will share a copy instead of having a copy in each session. Some builders such as Localized Resource will create read-only variables. Similarly, Variables that are marked as request scoped will not be stored in the session.

Note that in Portal, each portlet gets its own session, and once a portlet session is created, it is not released until the entire Portal session is released. This means that if you have a large number of portlets spread across many Portal pages, they can potentially all end up in session. Therefore when you have a large number of portlets, it's especially critical to keep Variables in session as small as possible.

Use caching across users when feasible.

The Cache Control builder lets you cache the results of an operation across users. The results are shared for all users of the same model and profile, so this mechanism is not suitable for user-specific queries or results.

The Service Operation and Lookup Table builders also have options for caching results, and both use the functionality of the Cache Control builder under the covers.

Note that when you are using the Cache Control builder, you have to select a method that returns the data to cache, and that method should not have any important side effects.

If you cache a method or service operation that takes input arguments, the Cache Control builder will automatically keep a separate cache entry for each unique set of input values. Therefore you should use it cautiously for methods that take arguments, since you could potentially end up with a large number of cache entries.

The data cached with these builders is associated with a specific generated WebApp. In other words, each profile will in general have its own cache. This is important in some cases. However, if you have a scenario where data can be cached across profiles, it may be best to implement the caching in a non-profiled model (perhaps a service provider model), so that the cached data can be shared across all users.

 
Avoid deploying a large number of Portlet Factory WAR files.

Instead, combine portlets in a small number of WAR files (ideally, one or two). You can have as many portlets as you want in a single WAR file. Each WAR has overhead of 8MB or more, so if you deploy 20 WARs, you could use perhaps 150MB - 200MB extra memory, when compared with using a single WAR. Also, keep in mind that Portlet Factory caching cannot be shared across WARs. For example, if you have a service provider with a cached operation returning several MB of data, each WAR will have its own copy of that data.
Use the latest version and fixpack of Portlet Factory.

Each Portlet Factory release typically includes additional performance and memory optimization work. By using the latest release you can get the benefit of all Portlet Factory product improvements related to performance. Check to make sure you have the latest fixpack also.
Use Portlet Factory's built-in logging tools to look at performance-related information.

There are some useful logging tools built into Portlet Factory that can help look at performance. For example:

• Server stats. Server statistics are recorded in a log file in deployed WAR files whenever the server is running. The server stats logging records statistics about requests to the Portlet Factory application every five minutes. This can provide valuable information about the "health" and performance of the application, and is most useful for looking at servers under load, either in testing or production. Since server stats logging does not add any significant performance overhead, it is recommended that they are left enabled (as they are by default). The server statistics are written to WEB-INF/logs/serverStats.txt.
• Model action tracing. This feature lets you see all the actions that take place during server requests to a model. It shows the sequence of operations, such as calls to methods and page display, with elapsed time for each. This is most useful for looking at application behavior for a single user during testing.
• Session size diagnostics. This feature lets you see all the variables that are in use by a running application. It can help you find areas where you may be putting a lot of data into session, for example, by retrieving a large data set. Like model action tracing, this is most useful for targeted testing with a single user.

This tech note has more information on these features: Techniques to enhance WebSphere Portlet Factory application peformance

For analyzing overall Java heap memory use, the verbose GC logging should be enabled in the application server. This log file (native_stderr.log) can be conveniently viewed with the PMAT tool available on developerWorks (http://www.alphaworks.ibm.com/tech/pmat).

Use automated load testing tools to validate the performance and capacity of your application.

In order to find the capacity for a particular application, for example to know how many concurrent users can be accomodated on a single server, there is no substitute for automated load testing of the application. Every application is different, and it is often not possible to extrapolate between applications.
See the WebSphere Portal and WebSphere Application Server tuning guides for recommendations to optimize server performance.      

For WebSphere Portal, see the IBM WebSphere Portal Version 6.0 Tuning Guide
Be aware of the impact of Portlet Factory profiling features.

When you use profiling, each unique profile will generate its own "WebApp" object in memory, with metadata describing the pages, methods, variables, and so forth for the model. Each WebApp also has separate cached results when you use the Cache Control builder or the caching features of Service Operation and Lookup Table. Therefore you should avoid having a very large number of different profiles for a model. For example, if you allow individual users to edit profile values with the "Edit" mode in Portal, you might potentially end up with a separate WebApp for every user. See the next item for how to avoid this.

Also, when you are using caching, if you can cache data in an unprofiled model such as a service provider model, this is better than caching data in a profiled model, since you'll have a single shared cache instead of multiple entries. On the other hand, if you need profile-specific cache results, you can get them by caching in the profiled model
When using profiling for Edit (Personalize) support in Portal, use the "Execution Time" setting for profile entries.

When you allow users to edit individual values in "Edit" mode, you may have a potential situation where each user ends up with their own generated WebApp (see information above on the impact of profiling). The "Execution Time" setting for profile entries can, in some scenarios, allow you to avoid this, by having the developer indicate which profile values are simple runtime values that don't require a unique regen. This technique can also be valuable in other situations (in addition to Edit mode) where you may have a large number of unique profiles.

To edit this setting, open up the profile set by double-clicking the ".pset" file, then go to the "Entries" tab, select the desired profile entry and click the "Edit" button. The choice for Execution Time is at the bottom of that dialog box. Note that you can only use this choice when the associated profiled builder input can accept a runtime value such as a variable reference. It cannot be used when profiling a value which would result in different generated code, such as the filename for the Imported Page builder.
Utilize the capabilities of external systems when possible.

Instead of using a Method / LJO to categorize, sort, compute, or sum data from an external source, let the external data storage system to do the work for you. Usually sources such as databases are much better equipped to handle such tasks.