ShowTable of Contents
Introduction
Today almost all infrastructures are monitored, and almost all monitoring systems are able to tell us, for instance, how much CPU, memory, hard disk, and bandwidth is being used. However, only a few are able to get measurements from the Java
TM Virtual Machine (JVM).
If our scope is to capture specific IBM® WebSphere® Portal measurements from our JVM, using the wsadmin tool, we can use the IBM WebSphere Application Server 8.0 Administrator Thin Client to run a jython script for collecting the information we seek.
The scenario
My idea is configure a standalone machine to collect measurements, as illustrated in the schema in figure 1, in which the JUH-Collector can connect to any JVM through a Simple Object Access Protocol (SOAP) port and collect the measurement in a file.
Figure 1. Configuration schema
JUH is the friendly name of this JVM Monitoring Project. Juh was an Apache warrior whose name means "He sees ahead", a brief description of which can be found in Wikipedia. The project was named after him because I wanted to create monitoring software that "sees ahead".
Solution
To create our JUH Monitoring project we must first install the Administration Thin Client on a standalone machine, as outlined in the steps below. After the Administration Thin Client is working, you must create a Jython script to collect measurement; in this example, the script collects the data in a .csv file, but it's not difficult to convert the output into other formats.
Here is the procedure, per the relevant steps in the product documentation topic, “
Using the Administration Thin Client.”
1. If the IBM
JVM Software Development Kit (SDK) is not installed, then install it on the JUH Machine.
2. Verify that python is installed on the JUH Machine.
3. Copy the Administration Thin Client JAR files (com.ibm.ws.admin.clientXXX.jar) from an IBM WebSphere Application Server environment to the JUH Machine, for example,
/opt/juh/thclient . The com.ibm.ws.admin.client_xxx.jar Administration Thin Client JAR file is located in one of the following locations:
• the AppServer/runtimes directory
• the AppClient/runtimes directory, if you optionally selected the Administration Thin Client when you installed the application client.
4. Copy the messages directory from the
app_server_root/properties directory to the
/opt/juh/thclient/properties directory.
5. To use in a secured environment, perform the following steps:
a) Copy the com.ibm.ws.security.crypto.jar file from either the AppServer/plugins directory or the AppClient/plugins directory and put it in the
/opt/juh/thclient directory.
b) If you are using the IPC connector, optionally copy the ipc.client.props file from the AppServer/profiles/
profileName/properties or the AppClient/properties directory and put it in the
/opt/juh/thclient/properties directory. Alternatively, you can set the properties in the ipc.client.props file programmatically in your Java code.
c) If you are using the SOAP connector, optionally copy the soap.client.props file from the AppServer\profiles\
profileName\properties directory and put it in the
/opt/juh/thclient/properties directory. Then, enable the client security by setting the com.ibm.CORBA.securityEnabled property to true. Alternatively, you can set the properties in the soap.client.props file programmatically in your Java code.
d) Copy the wsjaas_client.conf files from either the AppServer\profiles\
profileName/properties directory or the AppClient/properties directory, and put them in the
/opt/juh/thclient/properties directory.
e) Copy or generate the ssl.client.props file from either the AppServer\profiles\
profileName/properties directory or the AppClient/properties directory and put it in the
/opt/juh/thclient/properties directory.
f) Copy the key.p12 and trust.p12 files from AppServer\profiles\
profileName\
etc directory and put it in the
/opt/juh/thclient/etc directory.
g) To complete this task, copy the file to your thin client directory or run a script to generate the file. For more information, see the following product documentation topics:
Attention: This file contains the user.root property. You must modify the value to your thin client directory, for example, /opt/juh/thclient/
Sample scripts
All the scripts are deployed in /opt/juh/script. The script in listing 1 can work in two modes: In the first mode the script starts a new connection every time, starting and stopping the JVM for every collection; whereas, in second mode the script starts a JVM, opens a SOAP connection, and performs the polling without closing the main connections. If you want to measure the collected data with a short delay, you should use the second working modality.
Listing 1. Sample jython collector
import re, sys, time, os, java
if len(sys.argv)==0:
print __doc__
sys.exit(1)
# timing of loop to collect measure
loopTime=int(sys.argv[0])
print "Loop Time: " + str(loopTime)
perfObject = AdminControl.completeObjectName('type=Perf,*') #@UndefinedVariable - comes from WAS
if perfObject is None or perfObject == "":
print "Can not retrieve the performance MBean. Make sure PMI (performance metrics) is enabled on the server"
perfOName = AdminControl.makeObjectName(perfObject) #@UndefinedVariable
objNameString = AdminControl.completeObjectName('WebSphere:type=Server,*')
processName = AdminControl.getAttribute(objNameString, 'name')
# Output file for collections
OutFile = '../Collection/SysMetrics.' + processName + "_" + time.strftime("%d%m%Y", time.localtime()) + '.csv'
print "output Stored in: " + OutFile
writeHeader=1
try:
if os.path.isfile(OutFile):
writeHeader=0 # do not write a header to a file that is being appended
f_out=open(OutFile,"a")
except:
print "Error opening file %s\n%s" % (OutFile,sys.exc_info()[1])
sys.exit(2)
if len(sys.argv) > 1:
print "Argomento 2: " + sys.argv[1]
if sys.argv[1]=='-l':
# list objects with stats
print "Listing objects ..."
for a in AdminControl.queryNames('*').split('\n'): #type=ConnectionPool, @UndefinedVariable
try:
config=AdminControl.invoke_jmx(perfOName, 'getConfig', [AdminControl.makeObjectName(a)], ['javax.management.ObjectName'])
if config is not None:
print f_out, a
except:
pass
print "done."
sys.exit(0)
# Enable PMI data.
AdminControl.invoke_jmx (perfOName, 'setStatisticSet', ['all'], ['java.lang.String'])
WASobjects={}
# definition object
WASobjects['JVM']=AdminControl.makeObjectName(AdminControl.completeObjectName('type=JVM,*'))
WASobjects['MQThread'] = AdminControl.makeObjectName(AdminControl.completeObjectName('name=WebContainer,type=ThreadPool,*'))
WASobjects['System'] = AdminControl.makeObjectName(AdminControl.completeObjectName('name=SystemMetrics,*'))
namelist=['Date','Time']
# list all possible data to collect
for obj in WASobjects.keys():
statconfig=AdminControl.invoke_jmx(perfOName, 'getConfig', [WASobjects[obj]], ['javax.management.ObjectName'])
for d in statconfig.listAllData():
namelist.append(obj+d.name)
if writeHeader:
print f_out,",".join(namelist)
def run(cmd):
process = java.lang.Runtime.getRuntime().exec(cmd)
stdoutstream = ''
errorstream = ''
running = 1
while running:
while process.getInputStream().available(): # > 0:
stdoutstream += chr(process.getInputStream().read())
while process.getErrorStream().available(): # > 0:
errorstream += chr(process.getErrorStream().read())
try:
process.exitValue()
running = 0
print "done..."
return (stdoutstream, errorstream)
except java.lang.IllegalThreadStateException, e:
#print "running..."
#pass
time.sleep(0.1)
#return stdoutstream, errorstream
def get_value(s):
value=1
if str(s.getClass())=='com.ibm.ws.pmi.stat.BoundedRangeStatisticImpl':
value=s.current
elif str(s.getClass())=='com.ibm.ws.pmi.stat.CountStatisticImpl':
value=s.count
elif str(s.getClass())=='com.ibm.ws.pmi.stat.DoubleStatisticImpl':
value=s.double
elif str(s.getClass())=='com.ibm.ws.pmi.stat.TimeStatisticImpl':
value=s.count
elif str(s.getClass())=='com.ibm.ws.pmi.stat.RangeStatisticImpl':
value=s.current
elif str(s.getClass())=='com.ibm.ws.pmi.stat.AverageStatisticImpl':
value=s.count
else:
print s.getClass()
return value
print "Pulling WebSphere statistics...Press Ctrl-C to interrupt"
cycle=0
while cycle<=1:
try:
statshash={}
for t in namelist:
if t=="Date":
statshash[t]=time.strftime("%m/%d/%Y", time.localtime())
elif t=="Time":
statshash[t]=time.strftime("%H:%M:%S", time.localtime())
else:
statshash[t]=""
for obj in WASobjects.keys():
stats=AdminControl.invoke_jmx(perfOName, 'getStatsObject', [WASobjects[obj], java.lang.Boolean ('false')], ['javax.management.ObjectName', 'java.lang.Boolean'])
for s in stats.statistics:
statshash[obj+s.name]=get_value(s)
print f_out,",".join([str(statshash[v]) for v in namelist])
except:
print "%s. Encountered a glitch: %s, %s" % (time.strftime("%m/%d/%Y %H:%M:%S", time.localtime()),sys.exc_info()[0],sys.exc_info()[1])
if loopTime == -1:
cycle=cycle+1
if cycle<1:
time.sleep(loopTime) # collection delay
To simplify the launch of the script, I used the script in listing 2 to create a bash file to start the collections. This script updates the properties file for you.
Listing 2. Script to create bash file
#
# Script to use Parametric method to call a Collection data
# menu item
#
inputData()
{
echo -n "Enter process Id :"
echo "(name of the process make the collection)"
read procesId
echo -n "Enter ip or Fqdn of Server to Collect : "
read hostIp
echo -n "Enter Port where the JVM listen : "
read soapPort
echo -n "Enter Admin Id: "
read wasAdmin
echo -n "Enter Password : "
read wasPwd
export wasHost=$hostIp
export wasSoapPort=$soapPort
export wasAdminId=$wasAdmin
export wasAdminPwd=$wasPwd
}
# make script to one time run, the script collect until user press ctrl-C in terminal session.
oneTime()
{
echo
echo
echo "Start one time WsAdmin Collection" ;
echo
inputData
echo -n "Enter delay of cycle (in seconds) : "
read timeCycle
echo "press any key to start......"
read ss
scriptId=/opt/juh/Script/$procesId.sh
echo cd /opt/juh/Script/ > $scriptId
echo /opt/juh/adminClient/wsthinadmin.sh -lang jython -host $hostIp -port $soapPort -user $wasAdmin -password $wasPwd -f /opt/juh/jython/OneCollect.py $timeCycle $scriptId
chmod 771 $scriptId
echo "Configure Wsadmin Thin Client"
python /opt/juh/jython/setSoapClient.py /opt/juh/adminClient/properties/soap.client.props
echo " Start Collection"
$scriptId
}
#
# make script to schedule run, the script collects one time any run.
pulTime()
{
echo
echo
echo "Start puling WsAdmin Collection" ;
echo
inputData
echo -n "Enter delay of cycle (in seconds) : "
read timeCycle
echo "press any key to start......"
read ss
scriptId=/opt/juh/Script/skd_$procesId.sh
echo cd /opt/juh/Script/ > $scriptId
echo /opt/juh/adminClient/wsthinadmin.sh -lang jython -host $hostIp -port $soapPort -user $wasAdmin -password $wasPwd -f /opt/juh/jython/collectSkd.py $procesId -1 $scriptId
chmod 771 $scriptId
}
#
tester()
{
echo
inputData
python ./writefile.py /opt/juh/adminClient/properties/soap.client.props
}
#
cd /opt/juh/Script/
clear
while :
do
clear
echo "-------------------------------------"
echo " Main Menu "
echo "-------------------------------------"
echo "[1] Start one time WsAdmin Collection"
echo "[2] Start puling WsAdmin Collection"
echo "[5] Exit/Stop"
echo "======================="
echo -n "Enter your menu choice [1-5]: "
read yourch
case $yourch in
1) oneTime ; exit 0 ;;
2) pulTime ; exit 0 ;;
5) exit 0 ;;
*) echo "Opps!!! Please select choice 1,2 or 5";
echo "Press a key. . ." ; read ;;
esac
done
The script creates a bash to be scheduled. The bash file is stored in /opt/juh/Script and its name is skd_$procesId.sh If you chose to work in the first mode, its name is $procesId.sh, and if you chose to work in the second mode, then $procesId is the value you input.
Now you must schedule it, using crontab. The script in listing 3, which is used to set the SOAP properties, must be saved in
/opt/juh/script/jython/setSoapClient.py
Listing 3. Script to set SOAP properties
import sys,os
fileb=sys.argv[1]+".bak"
filet=sys.argv[1]
print fileb
print filet
os.rename (filet,fileb)
f = open(fileb, "r") # read the file by lines
lines = f.readlines()
f.close()
wasUser= os.environ['wasAdminId']
wasPwd= os.environ['wasAdminPwd']
ft = open( filet, "w" ) #as target:
for line in lines:
for word in line.split():
if line == "\n":
print ">"+line+ "<"
break
if line[0] == '#':
ft.write(line)
break
if word[:25] == 'com.ibm.SOAP.loginUserid=':
ft.write(word[:25]+wasUser+"\n")
else:
if word[:27] == "com.ibm.SOAP.loginPassword=":
ft.write(word[:27]+wasPwd+"\n")
else:
if word[:4] == "com.":
ft.write(word+"\n")
ft.close()
Table 1 is a small sample of collections, all of which are stored in
/opt/juh/Collection in a file named SysMetrics.<processName>_<date>.csv, where
is the name you input using the script in listing 2 above, and the format of the file is as shown in the table.
Table 1. Sample collections
Date | Time | CPU cycle usage | CPU cycle since server start | Free sys mem | JVM HeapSize | JVM
free mem | JVM
used mem | JVM
up time |
10/12/12 | 23:21:49 | 4 | 527 | 1775744 | 262846 | 165887 | 96959 | 528 |
10/12/12 | 23:22:09 | 0 | 547 | 1775704 | 262846 | 164238 | 98608 | 548 |
10/12/12 | 23:22:29 | 0 | 567 | 1774960 | 262846 | 162669 | 100177 | 568 |
10/12/12 | 23:22:49 | 4 | 586 | 1774504 | 262846 | 115123 | 147722 | 588 |
10/12/12 | 23:23:09 | 2 | 606 | 1773668 | 262846 | 113537 | 149309 | 608 |
10/12/12 | 23:23:29 | 3 | 626 | 1773588 | 262846 | 111965 | 150881 | 628 |
10/12/12 | 23:23:49 | 4 | 646 | 1771280 | 262846 | 110422 | 152424 | 648 |
10/12/12 | 23:24:09 | 3 | 666 | 1773644 | 262846 | 108792 | 154054 | 668 |
10/12/12 | 23:24:29 | 0 | 668 | 1773700 | 262846 | 107247 | 155599 | 688 |
10/12/12 | 23:24:49 | 0 | 706 | 1773656 | 262846 | 105706 | 157140 | 708 |
10/12/12 | 23:25:09 | 0 | 726 | 1773772 | 262846 | 104064 | 158782 | 728 |
10/12/12 | 23:25:29 | 0 | 746 | 1773736 | 262846 | 102539 | 160306 | 749 |
10/12/12 | 23:25:49 | 0 | 766 | 1773308 | 262846 | 100927 | 161918 | 769 |
10/12/12 | 23:26:10 | 0 | 786 | 1773296 | 262846 | 99347 | 163499 | 789 |
10/12/12 | 23:26:30 | 0 | 806 | 1773560 | 262846 | 97748 | 165097 | 809 |
10/12/12 | 23:26:50 | 0 | 826 | 1773624 | 262846 | 96176 | 166670 | 829 |
10/12/12 | 23:27:10 | 0 | 846 | 1773600 | 262846 | 94538 | 168307 | 849 |
10/12/12 | 23:27:30 | 0 | 866 | 1773620 | 262846 | 92991 | 169854 | 869 |
10/12/12 | 23:27:50 | 0 | 868 | 1773664 | 262846 | 91461 | 171384 | 889 |
10/12/12 | 23:28:10 | 0 | 906 | 1773660 | 262846 | 89887 | 172958 | 909 |
10/12/12 | 23:28:30 | 0 | 926 | 1773600 | 262846 | 88309 | 174536 | 929 |
10/12/12 | 23:28:50 | 0 | 946 | 1773888 | 262846 | 86794 | 176052 | 950 |
10/12/12 | 23:29:10 | 0 | 966 | 1773932 | 262846 | 85203 | 177642 | 970 |
Now with this data we can create a graph to show how our JVM works, as shown in figure 2.
Figure 2. JVM Used Memory

Conclusion
Using the technique described here, you can collect data from various WebSphere sources and manage it in the best way. You can also change the script so that the data collected can be sent to a monitoring system, or you can add scripts that create graphs representing the system's performance in real time. In this way, you have the opportunity to effect changes proactively, instead of having to chase error situations after the damage is done.
Tell us what you think
Please visit this link to take a one-question survey about this article:
Resources
About the author

Andrea Fontana, a 2013 IBM Champion for WebSphere, currently works as a System Architect, defining, organizing, and configuring complex IBM product-based solutions. In particular he works with WebSphere Portal and its collaborative environment including Domino 8.0.x, 8.5, IBM Connections 3.0.1, Lotus Quickr 8.0.x, and IBM Sametime, with respect to setting up SSO Kerberos integration solutions and configuring systems with a r-proxy solution with SSL integration. His past experience includes roles as an Application Developer, Database Administrator, and Project Manager in a wide variety of business applications. He graduated from the ITIS Zuccante C., Mestre (Venice), specializing in Industrial Electronics. You can reach Andrea at andrea.fontana@factor-y.com or a.fontana@net2action.com.