The deployment model of this scenario will be like bellow
In this model we have two rings, Cassandra ring and ZooKeeper ring. Since we are shipping a cassandra server and zoo keeper server with Message broker product we can use them and create a cassandra ring and a zoo keeper ring in the cluster.
We have three Message Broker servers in three different hosts as and all of them are having a cassandra server in it
First we need to configure the HostName in carbon.xml file which is located in "wso2mb-2.0.1/repository/conf/" directory. Specify the ip of the host as follows. You need to configure this for all the three servers.
<!-- Host name or IP address of the machine hosting this server e.g. www.wso2.org, 192.168.1.10 This is will become part of the End Point Reference of the services deployed on this server instance. --> <HostName>192.168.0.100</HostName>
Then We need to configure cassandra servers in these MB servers to form a cassandra ring. For that we need to change the following entries in the "Cassandra.yaml" file which is located in the "wso2mb-2.0.1/repository/conf/etc" directory.
# seeds is actually a comma-delimited list of addresses. # Ex: "<ip1>,<ip2>,<ip3>" - seeds: "192.168.0.101,192.168.0.102"
listen_address: 192.168.0.102
rpc_address: 192.168.0.103
Configuring Zookeeper Cluster
Then we need to configure the Zoo Keeper Servers to create a ring within among all broker nodes. We can do it by adding client port address to the "zoo.cfg" which is located at "wso2mb-2.0.1/repository/conf/etc " .
After modifying the file , it will look like bellow.
tickTime=2000 dataDir=repository/data/zookeeper clientPort=2181 start_zk_server=true clientPortAddress=192.168.0.101 server.1=192.168.0.100:2888:3888 server.2=192.168.0.101:2888:3888 server.3=192.168.0.102:2888:3888
Above zoo.cfg file is for the server 192.168.0.101. You can copy the same zoo.cfg file to other two servers and change the entry "clientPortAddress" to the ip of that particular server.
Then you need to create file "myid" in the data directory (repository/data/zookeeper) of each server. In that file you need to specify the id of the server.
eg: Id of the the server 192.168.0.100 will be "1". You need only to have the number 1 in that myid file.
You can find more information on this in zookeeper clustering setup guide
You need to do above configuration in each of the message broker servers appropriately.
Then you need to do the following configuration changes in each Message broker nodes
Modify qpid-config.xml file which is located at : /repository/conf/advanced directory as bellow
<clustering> <enabled>true</enabled> <OnceInOrderSupportEnabled>false</OnceInOrderSupportEnabled> <externalCassandraServerRequired>false</externalCassandraServerRequired> <coordination> <!-- Apache Zookeeper Address --> <ZooKeeperConnection>192.168.0.100:2181,192.168.0.101:2181,192.168.0.102:2181</ZooKeeperConnection> <!-- Format yyyy-MM-dd HH:mm:ss --> <ReferenceTime>2012-02-29 08:08:08</ReferenceTime> </coordination>
Now you can start all three Message Broker nodes.
The WSO2 Oxygen Tank Library : The Library contains articles, tutorials, presentations, and other knowledge base items published on The Oxygen Tank on different styles of deployments.