Addendum: Performance tuning for Mashup Center 2.0.0.2 

Contents: IBM Mashup Center Performance Tuning Guide
Download this content as a PDF file.

Understanding the benchmark environment

The deployment scenario for Mashup Center that was used by the IBM performance team is similar to the Mashup Center 2.0 deployment scenario. The scenario consists of the following five separate servers connected by 1000 Mbps Ethernet:

• WebSphere Edge Components Version 6.1 to act as the reverse proxy server
• WebSphere Edge Components Version 6.1 to act as the reverse proxy server
• A Mashup Center server deployed on AIX 6.1, which consisted of the following components:
• IBM HTTP Server 6.1
• WebSphere Application Server instance of Lotus Mashups
• WebSphere Application Server instance of MashupHub
• Oracle database server for Lotus Mashups
• Oracle database server for MashupHub
• IBM Tivoli Directory Server

These are the key differences between the Mashup Center 2.0 and 2.0.0.2 deployment scenarios:

• The Mashup Center server is deployed on AIX6.1 instead of Windows 2003.
• The database server is Oracle instead of IBM DB2.

Here is a graphical image of the Mashup Center 2.0.0.2 deployment topology:

Tuning the AJAX proxy

The implementation of the AJAX Proxy has changed from Mashup Center 2.0 to 2.0.0.2. The 2.0.0.2 implementation has no HTTP connection pool at the application level. Compared to 2.0, the overhead of the HTTP connection opening and closing is larger. As a result, in order to improve performance, we recommend that you do the following steps to enable the system level HTTP connection pool and set an appropriate pool size. This tuning works applies to all operating systems.

1. Log into the WebSphere Application Server administrative console.
2. Click Servers > Server Types > WebSphere application servers > Java Virtual Machine.
3. In the Application servers section in the main pane, click server1.
4. In the Server Infrastructure section, click Java and Process Management > Process definition.
5. Under Additional Properties, click Java Virtual Machine.
6. In the Generic JVM arguments field, add the following line:

-Dhttp.maxConnections=200 -Dsun.net.http.errorstream.enableBuffering=true

See the following image for an example:

Tuning the data source for AIX

On an AIX 6.1 platform, the large reap time of the data source may lead to a spike in CPU usage under a heavy workload. To solve this usage issue, do the following steps to set the preferred values for reap time and aged timeout:

1. Log into the WebSphere Application Server administrative console.
2. Click Resources > JDBC > Data sources.
3. In the Data sources section in the main pane, click mashupDS.
4. In the Additional Properties section, click Connection pool properties.
5. In the Reap time field, set the value to 0.
6. In the Aged timeout field, set the value to 0.

Now the fields should look like this:

Tuning JVM initial and maximum heap size for AIX

The value of the JVM heap size is directly related to the amount of physical memory on the system. Never set the JVM heap size larger than the physical memory on the system. When setting the JVM heap size, be sure to follow the recommendations for Mashup Center 2.0. In our performance tuning, we took the following steps to set the value of the JVM heap size to 1400MB:

1. Log into the WebSphere Application Server administrative console.
2. Click Servers > Server Types > WebSphere application servers > Java Virtual Machine.
3. In the Application servers section in the main pane, click server1.
4. In the Server Infrastructure section, click Java and Process Management > Process definition.
5. Under Additional Properties, click Java Virtual Machine.
6. In the Maximum heap size field, set the value to 1400.

Tuning JVM heap for large pages

Large pages can reduce the CPU overhead needed to keep track of heap size. With this setting, we have seen 5% throughput improvement in our measurements.

Although this setting improves performance on Windows, we did not set it for our measurements because Mashup Center does not start reliably when –Xlp is set and sometimes requires a system reboot before starting or restarting a JVM. However, if you want to set the JVM heap size, do the following steps on Windows platforms:

1. Log into the WebSphere Application Server administrative console.
2. Click Servers > Server Types > WebSphere application servers > Java Virtual Machine.
3. In the Application servers section in the main pane, click server1.
4. In the Server Infrastructure section, click Java and Process Management > Process definition.
5. Under Additional Properties, click Java Virtual Machine.
6. In the Generic JVM arguments field, add the following line: -Xlp

Tuning JVM large pages on AIX

To use JVM large pages on AIX, do the following steps to add support for large pages:

1. Use the following values to allocate 4GB of RAM as large pages (16MB). We chose this amount based on the amount of physical memory in our systems, which was 8GB. You may need to adjust these values for systems with different amounts of physical memory.
   
   vmo -r -o lgpg_regions=256 -o lgpg_size=16777216
   bosboot -ad /dev/ipldevice
   reboot -q
   vmo -p -o v_pinshm=1
   chuser capabilities=CAP_BYPASS_RAC_VMM,CAP_PROPAGATE $USER
   
   
2. Do the following steps to add the JVM heap size:
   
   1. Log into the WebSphere Application Server administrative console.
   2. Click Servers > Server Types > WebSphere application servers > Java Virtual Machine.
   3. In the Application servers section in the main pane, click server1.
   4. In the Server Infrastructure section, click Java and Process Management > Process definition.
   5. Under Additional Properties, click Java Virtual Machine.
   6. In the Generic JVM arguments field, add the following line: -Xlp
3. Still in the administrative console, do the following steps:
   
   1. Click Servers > Server Types > WebSphere application servers > Java Virtual Machine.
   2. In the Application servers section in the main pane, click server1.
   3. In the Server Infrastructure section, click Java and Process Management > Process definition.
   4. Under Additional Properties, click Environment Entries.
   5. Click New.
   6. In the General Properties section, in the Name field, type EXTSHM. In the Value field, type OFF. Note that when the value is set to ON, it prevents the use of large pages.
   7. Click OK. Now you environment entries page should show your new entry, for example:
      
      
      
4. Restart the WebSphere Application Server server. To verify if large pages are being used, run the following AIX command: vmstat -l 1 5. Now check the alp column, which is the active large page that is currently being used. The value should not be 0 if large pages are being used.

You can also use large pages for memory areas other than the Java heap. These other memory areas include the program text (machine instructions), data area (all other data outside of the Java heap), and stack. For the AIX operating system, one mechanism for using large pages for other data areas is by using environment variables, for example:

export LDR_CNTRL="TEXTPSIZE=64K@DATAPSIZE=64K@STACKPSIZE=64K"

Running Java after setting the environment variable, as in the example above, causes 64 KB pages to be used for all of the text, data, and stack areas. More details about changing the page sizes used by programs are available in the AIX documentation.

Tuning AIX networking

In our measurement environment, we changed the following network tuning parameters on all the AIX systems. For AIX, use the no command to set the following TCP layer parameters. These take effect immediately to improve the network layer performance in high volume environments.

• /usr/sbin/no -o tcp_sendspace=65536
• /usr/sbin/no -o tcp_recvspace=65536
• /usr/sbin/no -o udp_sendspace=65536
• /usr/sbin/no -o udp_recvspace=65536
• /usr/sbin/no -o somaxconn=10000
• /usr/sbin/no -o tcp_nodelayack=1

To make these changes permanent across reboots, add these statements to the /etc/rc.net file.

Previous topic: Understanding general performance characteristics of the catalog, feed generators, and data mashup builder