Site analytics and optimization 

Table of Contents | Previous Page | Next Page

---

Portal Analytics is the process of collecting, processing, and reporting portal usage data.

Often, enterprises using a portal for their external web site faced with questions such as:

• "What do our users really do with the portal?"
• "Will our portal be able to deal with evolving user needs?"
• "What are most users doing on the Portal?"
• "How should I plan for the growth of my Portal?"
• "What’s the most popular search?"
• "Where are my users coming from?"
• "What are my customers downloading?"
• and so on

Site Analytics help you understand how your site is being used, effectiveness of advertising campaigns, page or portlet popularity, measuring load such as page views per second and unique logins per hour, modeling real user behavior for capacity and performance tests, and more.

Site Analytics enables customers to provide a more personalized experience to their clients by deploying a more dynamic web solution. Site Analytics helps in identifying ways to better target content. For example, if a particular page in the portal site is being accessed more or users are directly going to a page often, should that page be the home page or that content be moved on to the home page. It increases revenue with better targeting and can decrease cost with automatic tuning. Site Analytics also increases customer satisfaction and reduces testing costs with better designs. Site Analytics allows you to measure the success of a portal, predict the demand to a portal in the future and pro-actively plan for adapting to the community’s needs, rather than being hit without warning.

So, customers are willing to buy products that are specialized and tested in Web site analysis. Integration with site analyzer tools are performed by creating reports based on the portal site analyzer logs or manually imbedding tags into portlets and themes. Hence, customers are expecting portal to provide the necessary data and offer seamless integration with each site analytics product.

Site Analytics gathers data on the following aspects of a portal, that can then be fed to an Analytic tool to analyze how a site is being used:

• Users requesting pages, incl. contained Portlets
• Session activities (login, logout, timed out, login failed)
• Page management (creating, updating, deleting a page)
• User Management actions (creating, updating, deleting users and groups)

Analytics are measured in one of two ways:

1. Server-side Analytics: It’s a fine-grained resource usage reporting method. It can be used offline and is ideal for historical analysis.
2. Active Site Analytics: It’s also called Client-side or "Web bugs". It’s a client side script-based and real-time reporting method, which collects cached content as well.

In general, active site analytics are recommended since they:

• have less impact on server resources,
• are easier to reconcile in a cluster since there is one log vs many,
• are easier to put in the instrumentation.

The subsequent sections will discuss the two analytic methods in detail.

Server side site analytics

Server-side Site Analytics, also known as logfile analysis, is a log file based approach. The site analysis infrastructure provided by the portal accommodates most scenarios and all the necessary data for site analytics. A typical server-side site analytics architecture is shown in the diagram below. User interactions and metadata are written to server logfiles (sa.log), hosted on the Portal Server. Logfiles can be recorded and stored for later offline processing. Logfile recording is activated via a simple configuration change in WebSphere Portal. A list of all the server-side loggers available is described in the table below.

Server side site analytics architecture

Information that can be derived from the analytics log includes, but is not limited to:

• Browsers of users visiting the portal
• Different operating systems as reported by the browser
• Different logins corresponding to authenticated users
• Robots and programmatic clients
• Pages that were requested, but not found
• Search engines, key phrases and key words
• The referring page a user came from
• Entry and exit URLs
• Virtual portal specific and WCM viewed usage statistics, since WebSphere Portal 6.1.0.1 is released

Site Analytic tools offered by third party vendors can be used to analyze the collected data and generate reports. Analytics engines are not part of the WebSphere Portal product offering. Earlier versions of Portal shipped with Tivoli Site Analyzer, which is now discontinued. Customers, who are still running WebSphere Portal version 5.1, can continue to use Tivoli Site Analyzer.

Server side loggers

WebSphere Portal provides fine grained usage information, but it will be up to the analytics engines to derive the information from the data in the logs. The recorded URLs are not real, clickable URLs, but they still provide the relevant information to find out which pages the users of the portal looked at. With the logfile based approach, only hits on the Portal Server are recorded, so cached page hits are not recorded. Portlets can report specialized business events by calling analytics log API, since WebSphere Portal version 6.1.0.1.

Logger	Purpose
SiteAnalyzerSessionLogger	Logs session events like login or logout
SiteAnalyzerUserManagementLogger	User and group management events like creating or deleting users and groups
SiteAnalyzerPageLogger	Page render events
SiteAnalyzerPortletLogger	Portlet render events
SiteAnalyzerJSRPortletLogger	Custom business events in standard Portlets (JSR 168, JSR 286)
SiteAnalyzerPortletActionLogger	Actions occurred in a Portlet
SiteAnalyzerApplicationActionLogger	Actions occurred in a Portlet application
SiteAnalyzerErrorLogger	Logs any errors

Analytics and reporting with WCM

The JSR 286 Web Content Viewer portlet implements detailed site analysis logging, which enables you to gather usage statistics of each instance of the WCM rendering portlet, since WebSphere Portal version 6.1.0.1 is released. It also offers syndication pending and deployment views as well as additional date fields.

The configuration process to enable WebSphere Portal Site Analyzer requires a simple change to a properties file. The logger that needs to be activated is SiteAnalyzerJSRPortletLogger logger. This will cause log information to be added to the site analytics log (sa.log) files. The site analytics log can then be formatted using a third party analytics tool in order to analyze the collected data. When the generated reports are shared with content authors, it will give them an opportunity to enhance their content to better target the content audience.

Active site analytics

Active site or Client-side analytics, also known as Page Tagging Analytics, is JavaScript injection based. JavaScript is embedded in the theme or skin of a page, so the JavaScript is also called a “web bug”. The web bug helps to gather data from portal perspective, where as 3rd party vendors or partners will provide the reporting function and analysis. IBM also provides a few sample JavaScript scripts out of the box, which can be modified to suit the needs or the partner supplied web bugs can be embedded in a portal page. A typical Active Site Analytics Architecture diagram is shown in the picture below.



The following diagram depicts a general scenario of how the data is collected in Active Site Analytics. When a client (browser) requests a portal page, portal returns page markup, including metadata markup. Client aggregates the metadata using JavaScript snippet or web bug embedded in theme/skin of the page. Client then sends aggregate data to the analytics server, which is typically installed on infrastructure, separate from portal server’s, e.g. via HTTP protocol. Analytics Server’s web server instance records the access log.

What gets recorded depends on the specific 3rd party client-side analytics solution. Typically recorded events are page display, listing of items on the page, download actions and links leaving the site. The injected JavaScript determines which data will be recorded and it requires proper planning. Some tools cover multiple requests per single page such as page, portlet and content. Injection into themes and skins allows reporting of page and Portlet views. Portlet business events are collected by adding JavaScript to the Portlet markup. Any rendering of the corresponding page in the user’s browser can be recorded, including cached pages. 3rd party Analytics software systems gather and process this data locally and generate reports.



Supported-tags or micro-format are used to annotate the page markup with the page name, portlet names and ids and themes / skins will be instrumented according to the micro-format for Active Site analytics logging in WebSphere Portal.

Microformat/ Supported Tags

Name	Description	Tagged as	Injected in
Page Title	Title of the page in portal default language.	asa.page.title	Theme
Portlet Window Title	Title of the portlet as delivered to the client	asa.portlet.title	Skin
Portlet Window ID	Unique identifier of the portlet	asa.portlet.id	Skin
WCM Content Querystring	Unique identifier of the WCM content item that is displayed in a portlet	asa.portlet.screen.id	WCM Rendering Portlet
WCM Content Title	May not be the same as the portlet window's title	asa.portlet.screen.title	WCM Rendering Portlet

Aggregator script supplied by the business partners will be injected into the portal page’s markup. The aggregator will iterate over all instances of the micro-format, then extract the relevant information and submit it to the external analytics server.
Aggregators are managed as part of the theme, including staging to production.

Instrumenting a theme for active site analytics

This section described the steps involved in instrumenting a theme for Active Site Analytics. Decide what data should be recorded and mark the data in the theme or skin JSPs:


Include aggregator with the page:


Adapt the aggregator, if required. Note that, dojo provides simple ways to access DOM data.

Differences between server side and active site analytics

Here is a comparison of Server-side versus Active Site Analytics. It’s important to note that, both approaches can be used with WebSphere Portal, either exclusively or in combination.

Server side

• User interactions and meta-data is written to server log files, hosted on the Portal Server.
• Log files can be recorded and stored for later offline processing.
• Analytics engines read these log files for collecting analytics data.
• WebSphere Portal provides fine grained usage information.
• Only hits on the Portal Server are recorded and for example, cached page hits are not recorded.
• Log file recording is activated via configuration in WebSphere Portal.
• No special coding needed for base reporting (pages, portlets, etc.)
• Portlets can report specialized business events by calling analytics log API, since portal version 6.1.0.1.

Active site

• JavaScript within the Portal page (e.g. theme / skin) notifies analytics server about user activity.
• Analytics data is collected online within the Analytics system, which gathers and processes this data outside the portal server and the browser.
• Up until WebSphere Portal v6.1, appropriate data and JavaScript needed to be injected manually into Portal. With v6.1.5, IBM provides the integration out of the box.
• The injected JavaScript determines which data will be recorded.
• Injection into themes and skins allows reporting of page and portlet views.
• Portlet business events are collected by adding JavaScript to the Portlet markup.
• Any rendering of the corresponding page in the user’s browser can be recorded, including the cached pages.

User experience optimization initiative

Today, IBM customers are demanding better integration between WebSphere® Portal Server and Web analytic tools. They use site metrics to capture and measure user activity primarily to understand end user needs, behaviours and site usability, so that portals can be better designed and targeted. In response to our customers’ requests, IBM created the IBM User Experience Optimization Initiative (UEOI) that teams up IBM with leading Web analytic software companies (Coremetrics, WebTrends, and Omniture) to seamlessly integrate their products with the WebSphere Portal family product line, OmniFind®, Quickr®, and IBM Lotus® Web Content Management. This is the first step in implementing our vision to provide an improved and more personalized experience to our clients by allowing them to deploy dynamic and best-of-breed website analytic solutions.

Three market-leading partners have signed strategic partnership agreements with IBM for this initiative -- Coremetrics, Omniture and WebTrends. Coremetrics is an IBM Partner and has an alliance with WebSphere Commerce. Omniture is a fast growing web analytic leader and a leading provider of online business optimization software. It has integrations with IBM WebSphere Commerce and WebSphere Portal. WebTrends is a well established analytics product among WebSphere Portal customers.

Future outlook and summary

WebSphere Portal will offer enhanced integration with Web Analytic vendor products, when version 7 is released in 2010. Future releases will also aim to create an administrative portlet, which will enable the user to choose the web analytic product he/she is using and add portlets with dashboard and content / grouping segmentation reports to provide portal and portlet views.

The long-term vision is to create a self-optimizing portal that automatically responds to the demand on the site and optimizes for better results.

Additional information and resources

Site Analytics properties on a portal server can be seen in file:
<profile>/PortalServer/config/SiteAnalyzerLogService.properties

WebSphere Portal 6.1.5 InfoCenter - Analyzing portal usage data
http://publib.boulder.ibm.com/infocenter/wpdoc/v6r1/topic/com.ibm.wp.ent.doc_v615/admin/sa.html

Using portal analytics with open-source reporting tools (developerWorks)
http://www.ibm.com/developerworks/websphere/techjournal/0609_liesche/0609_liesche.html

Adapting AWStats for IBM WebSphere Portal 6.0.x and virtual portals
http://www.ibm.com/developerworks/websphere/zones/portal/proddoc/dw-w-awstatsportal/

WebSphere Portal – IBM Site
http://www-3.ibm.com/software/genservers/portal/

WebSphere Portal Business Solutions Catalog:
http://catalog.lotus.com/wps/portal/portal

Websphere Portal Developer’s Zone
http://www-106.ibm.com/developerworks/websphere/zones/portal/

Product Documentation and WebSphere Portal Wiki
http://www-3.ibm.com/software/genservers/portal/library/
http://www-10.lotus.com/ldd/portalwiki.nsf

Education
http://www-3.ibm.com/software/genservers/portal/education/

---

Table of Contents | Previous Page | Next Page