[ Documentation Index ]

Starting external cassandra server and zoo keeper server and point all broker nodes to them

In this model you need to get downloaded following apache products

Deployment model will be like bellow

Cluster Setup Scenario 01

In this model we have a common cassandra server and a common zoo keeper server. Both of them can be in a same host or in different hosts. Eg: Lets say they are in two different hosts as bellow

  • Cassandra Server - 192.168.0.100
  • Zoo Keeper Server - 192.168.0.101

We have three Message Broker servers in three different hosts as

  • MB Server 01 - 192.168.0.102
  • MB Server 02 - 192.168.0.103
  • MB Server 03 - 192.168.0.104

We need to configure cassandra server to listen to all the broker nodes.For that we need to change the following entries in the "Cassandra.yaml" file which is located in the "apache-cassandra-1.0.1/conf/" directory.

  • Change the data storage directories of cassandra. Here by default it will point to the location "/var/lib/". But it will be better, if you can keep the data locally inside the folder where you have the server setup. So change it to "./repository/database" off all three entries as
  •               # directories where Cassandra should store data on disk.
                  data_file_directories:
                      - ./repository/database/cassandra/data
    
                  # commit log
                  commitlog_directory: ./repository/database/cassandra/commitlog
    
                  # saved caches
                  saved_caches_directory: ./repository/database/cassandra/saved_caches
    
              
  • Change listen address of the server. By default it is "localhost", change it to the ip address of the host where you have the cassandra server.
  •              listen_address: 192.168.0.100
              
  • Change RPC address of the server.By default it is "localhost", change it to the ip address of the host where you have the cassandra server.
  •              rpc_address: 192.168.0.100
              

Now save the cassandra.yaml file and you can start the cassandra server by executing the script "cassandra" with the option -f. Eg: "./cassandra -f"

Then we need to configure the Zoo Keeper Server to listen to all broker nodes. We can do it by renaming the "zoo_sample.cfg" file which can be found at "zookeeper-3.3.4/conf" folder to "zoo.cfg" and do the following modification in that file

  • Change the data directory to store data local to the installation directory
  • Add the entry "clientPortAddress" by providing the ip of the localhost

After modifying the file , it will look like bellow.

            # The number of milliseconds of each tick
            tickTime=2000
            # The number of ticks that the initial
            # synchronization phase can take
            initLimit=10
            # The number of ticks that can pass between
            # sending a request and getting an acknowledgement
            syncLimit=5
            # the directory where the snapshot is stored.
            dataDir=./../data
            # the port at which the clients will connect
            clientPort=2181

            clientPortAddress=192.168.0.101
        

Once you done the above changes, you can save the file and start the zookeeper server by executing the "zkServer.sh" script with the option start. eg: sh zkServer.sh start

Then you need to do the following configuration changes in each Message broker nodes

Modify qpid-config.xml file which is located at : /repository/conf/advanced directory as bellow

  • Enable clustering
  • Change the server of the Zookeeper
          <clustering>

          <enabled>true</enabled>
          <OnceInOrderSupportEnabled>false</OnceInOrderSupportEnabled>
          <externalCassandraServerRequired>true</externalCassandraServerRequired>

          <coordination>
              <!-- Apache Zookeeper Address -->
              <ZooKeeperConnection>192.168.0.101:2181</ZooKeeperConnection>
              <!-- Format yyyy-MM-dd HH:mm:ss -->
              <ReferenceTime>2012-02-29 08:08:08</ReferenceTime>
          </coordination>
      

Modify qpid-virtualhosts.xml file which is located at :/repository/conf/advanced directory as bellow

  • Change the connection string by pointing to the cassandra server
          <store>
              <class>org.wso2.andes.server.store.CassandraMessageStore</class>
              <username>admin</username>
              <password>admin</password>
              <cluster>ClusterOne</cluster>
              <idGenerator>org.wso2.andes.server.cluster.coordination.TimeStampBasedMessageIdGenerator</idGenerator>
              <connectionString>192.168.0.100:9160</connectionString>
          </store>
      

Now you can start all three Message Broker nodes.

Other Resources

The WSO2 Oxygen Tank Library : The Library contains articles, tutorials, presentations, and other knowledge base items published on The Oxygen Tank on different styles of deployments.