This sample explains how a cluster of servers can be monitored and can obtain the useful analysis out of the collected data. This sample concentrates on the statistical analysis of a clustered environment. Throughout the sample we'll assume a clustered system as follows;
There are two data centeres and each having two clusters; One is a ESB(Enterprise Service Bus) cluster and other one is the AS (Application Server) cluster.For the this kind of situation WSO2 BAM can be configured in such a way the request count, response count, etc for each cluster can be monitored. The below given diagram explains the architecture how WSO2 BAM can be connected the cluster in the above scenario.
As shown in the above diagram, the statistics from the each cluster is pushed in WSO2 BAM and the data is directly stored in the Cassendra data store. Analyzers can be written in order to process the gathered data by WSO2 BAM. The result of analyzers will be summarized and stored back in the cassendra data store. The summarized data can be presented in the gadget dashboard according to his/her requirement by using the Gadget-Ide in the WSO2 BAM.
Each of the above steps will be described in detail in the following steps.
Step 1 - Start BAM Server
Download the WSO2-BAM Server
Unzip the WSO2-BAM.zip in the location where ever you prefer
Go to WSO2-BAM-HOME/bin folder in comman prompt. And type
wso2server.sh (For ubuntu, linux, mac OS)
wso2server.bat (For Windows)
Step 2 - Sending events to BAM server from clusters
Pre-requisites: Apache-ant
Make sure you machine has apache-ant, if not please install the apache-ant.(http://ant.apache.org/)
Go to location from command prompt WSO2-BAM-HOME/samples/cluster-monitor
In this folder you can see the build.xml. Type 'ant' command from the command propmt. This will simultate the data center siutation as showed in the figure and will pump the events to BAM
The incoming data from the cluster to BAM will be as below;
According to this situation we need to create two indexes.
ClusterStreamSelector: For filter out the cluster stream data from huge amount of data stream stored in the cassendra data store
ClusterIndex: This index is needed to query the summarized data which is generated by the analyzer
At this moment we need to create the ClusterStreamSelector. Follow the below given steps to create the index.
Login to the BAM management console. The link for the management console will be there in the server startup log. You can click on it and log in to the management console. Or you can go to https://localhost:9443/ in your web browser, this will take you to the management console.
Go to Configure → Connection Parameters. There you can provide the connection parameter for the cassendra source. Default username, password is admin, admin. Click on Save button after filling out the connection paramters.
Go to Configure → BAM Database
Click on the Add Index.
Create Index named ClusterStreamSelector, which filters out the clusterStream, as shown below. And click Save.
Step 4 - Create Analyzer
In the analyzer you need to write the analytics such that it analyzes the gathered data by the BAM according to you requirement. In this sample we are concerned about the request count, response count, fault count and response time of each cluster. Therefore the analyzer sequence for this requirement will be,
The above given analyzer sequence is given in the WSO2BAM/Samples/cluster-monitor/ClusterSummarizer.xml. Follow the below given steps to ad dthe analyzer sequence to the BAM server.
Go to WSO2BAM/samples/cluster-monitor folder
Open the ClusterSummarixer.xml and copy the content
Go to Configure → Analyzer Framework
Click on 'Add sequence' and go to the source view of the analyzer sequence.
Paste the content from the ClusterSummarizer.xml in the step-2, and click on 'Save All' button.
After adding the summarizer, you can see the below summarized data in the server log.
Step 5 - Create Index (ClusterIndex)
This index is used to query the summarized data and create the gadget using that. This is similar to the step 3 where we created the ClusterStreamSelector.
Go to Configure → BAM Database
Click on Add Index
Create the index as in the below figure. Click on the 'Add column' to add the second column field.
Now you can see two index from the index table as below.
Step 6 - Create Gadget
In this section we cna see how to design your own gadgets according to your requirement. In this section we created bar chart, pie chart and table to visulaize the data created by BAM.
The below given steps will explain how to create the gadgets and visualize the data collected from the clusters of servers.
Go to Configure → Gadget IDE
In order to obtain the unique data center names, there is a web service in BAM. To communicate with web service, click on the 'Web Service (WSDL)' in the Gadget IDE.
You can see the WSDL of the web service mentioned above, in https://localhost:9443/services/IndexAdminService?wsdl2. Insert this WSDL file url in the gadget, as below. And click on 'load' button.
Select 'getIndexValues' for the field 'Operation' in the WSDL gadget.
Click on the Text and connect as below. Note the two Text gadgets have been used. One text field here will mention the indexName, and other one will mention the indexedColum which as two parameters which is required to to the 'getIndexValues' operation.
Now click on 'Combo box'. It'll immediately take to the design view, there you can resize and locate the combo box where ever you like. Then switch back to the 'DataFlow' view. Connect the 'return' node from the WSDL gadget to the 'text' node of the combo box as shown below.
This states the return value from the web service method is used to fill the combo box. Now go to the design view, there you can see the unique names of the data centers. (which is returned from the web service)
Click on the Cassendra sources. And fill it as the following and click on load button
Tablename: ClusterSummary (Where the summarizes data is stored. Refer the analyzer sequence <put> tag)
TableIndexName: ClusterIndex (Name of the index which was created to query the ClusterSummary)
Index: dataCenter(Name of the field for which the index was created)
Click on the lexical string and and connect as below shown.
Combo box-text → Cassendra source-from0
Combo box-text → lexicalString-text
lexicalString-text → cassendra source-to
Click on the bar chart. It'll take to the design view, locate the gadget as you prefer.
Switch to the Dataflow view. Connect
Cassendra Source-Cluster → lable-Bar Chart.
Cassendra Source-request count→ value-Bar Chart.
Switch to design view and see the barchart. You can select the data center from the combo box and the bar chart will show the values for that data center
Bar chart output of Grid sys data center
Bar chart output of NetVest data center
Go to the DtaaFlow view and Click on the 'Pie chart'. Then position the pie-chart in the design view as previouse steps and switch back to the DataFlow view. Connect,
Cassendra Source-Cluster → lable-Pie Chart.
Cassendra Source-fault count→ value-Bar Chart.
Now switch to the Design view and see the pie chart. You can select the data center as mentioned in the above step form the combo box.
Go to DataFlow view and Click on the table. Position the table in the design view. And rhen connect,
Table-rows → row-Cassendra Source.
Switch to the design view and see the table. Here also you can select the data center from combo box and see the respective data center values.
Final gadget view for NetVest data center
Final gadget view for GridSys data center
Now you can see the created gadget from the main dash board.
Step 7 - Publish the gadget to dashboard
Go to the settings in gadget ide side pane and click on the publish
Type a name of the gadget and click on publish. A URL will be popped out. Copy the URL.
Eg: /registry/resource/_system/config/repository/dashboards/gadgets/ClusterSummary.xml
Go to BAM server home page, and click on the BAM dashboard.
Click on the Add Gadget, and paste the URL copied which publishing the gadget.
Add click on the add gadget.
Now we have come to the end of the sample and now you know the way of generating the custom gadgets from BAM and monitoring your services in clustered environment.