org.apache.hadoop.hbase.replication
Class ReplicationZookeeper

java.lang.Object
  org.apache.hadoop.hbase.replication.ReplicationZookeeper

public class ReplicationZookeeper
extends java.lang.Object

This class serves as a helper for all things related to zookeeper in replication.

The layout looks something like this under zookeeper.znode.parent for the master cluster:

 replication/
  state      {contains true or false}
  clusterId  {contains a byte}
  peers/
    1/   {contains a full cluster address}
    2/
    ...
  rs/ {lists all RS that replicate}
    startcode1/ {lists all peer clusters}
      1/ {lists hlogs to process}
        10.10.1.76%3A53488.123456789 {contains nothing or a position}
        10.10.1.76%3A53488.123456790
        ...
      2/
      ...
    startcode2/
    ...
 

Nested Class Summary

class

ReplicationZookeeper.ReplicationStatusTracker Tracker for status of the replication

Constructor Summary

ReplicationZookeeper(Abortable abortable, org.apache.hadoop.conf.Configuration conf, ZooKeeperWatcher zk)
Constructor used by clients of replication (like master and HBase clients)

ReplicationZookeeper(Server server, java.util.concurrent.atomic.AtomicBoolean replicating)
Constructor used by region servers, connects to the peer cluster right away.

Method Summary

void	addLogToList(java.lang.String filename, java.lang.String clusterId) Add a new log to the list of hlogs in zookeeper
void	addPeer(java.lang.String id, java.lang.String clusterKey) Add a new peer to this cluster
boolean	connectToPeer(java.lang.String peerId) This method connects this cluster to another one and registers it in this region server's replication znode
java.util.SortedMap<java.lang.String,java.util.SortedSet<java.lang.String>>	copyQueuesFromRS(java.lang.String znode) This methods copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode)
void	deleteOwnRSZNode() Delete this cluster's queues
void	deleteRsQueues(java.lang.String znode) Recursive deletion of all znodes in specified rs' znode
void	deleteSource(java.lang.String peerZnode, boolean closeConnection) Delete a complete queue of hlogs
java.lang.String	getClusterId() Get the identification of the cluster
long	getHLogRepPosition(java.lang.String peerId, java.lang.String hlog) Get the position of the specified hlog in the specified peer znode
java.util.List<java.lang.String>	getListHLogsForPeerForRS(java.lang.String rs, java.lang.String id) Get the list of hlogs for the specified region server and peer cluster
java.util.List<java.lang.String>	getListOfReplicators() Get the list of the replicators that have queues, they can be alive, dead or simply from a previous run
java.util.List<java.lang.String>	getListPeersForRS(java.lang.String rs) Get the list of peer clusters for the specified server names
ReplicationPeer	getPeer(java.lang.String peerId) Helper method to connect to a peer
java.util.Map<java.lang.String,ReplicationPeer>	getPeerClusters() Get a map of all peer clusters
java.lang.String	getPeersZNode() Get the full path to the peers' znode
java.util.List<java.lang.String>	getRegisteredRegionServers() Get a list of all the other region servers in this cluster and set a watch
boolean	getReplication() Get the replication status of this cluster.
java.util.List<HServerAddress>	getSlavesAddresses(java.lang.String peerClusterId) Returns all region servers from given peer
static java.lang.String	getZNodeName(java.lang.String fullPath) Extracts the znode name of a peer cluster from a ZK path
ZooKeeperWatcher	getZookeeperWatcher() Get this cluster's zk connection
java.util.List<java.lang.String>	listPeersIdsAndWatch() List this cluster's peers' IDs
boolean	lockOtherRS(java.lang.String znode) Try to set a lock in another server's znode.
void	registerRegionServerListener(ZooKeeperListener listener)
void	removeLogFromList(java.lang.String filename, java.lang.String clusterId) Remove a log from the list of hlogs in zookeeper
void	removePeer(java.lang.String id) Remove the peer from zookeeper.
void	setReplicating(boolean newState) Set the new replication state for this cluster
void	writeReplicationStatus(java.lang.String filename, java.lang.String clusterId, long position) Set the current position of the specified cluster in the current hlog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReplicationZookeeper

public ReplicationZookeeper(Abortable abortable,
                            org.apache.hadoop.conf.Configuration conf,
                            ZooKeeperWatcher zk)
                     throws org.apache.zookeeper.KeeperException

Constructor used by clients of replication (like master and HBase clients)

Parameters:

conf - conf to use

zk - zk connection to use

Throws:

java.io.IOException

org.apache.zookeeper.KeeperException

ReplicationZookeeper

public ReplicationZookeeper(Server server,
                            java.util.concurrent.atomic.AtomicBoolean replicating)
                     throws java.io.IOException,
                            org.apache.zookeeper.KeeperException

Constructor used by region servers, connects to the peer cluster right away.

Parameters:

server -

replicating - atomic boolean to start/stop replication

Throws:

java.io.IOException

org.apache.zookeeper.KeeperException

Method Detail

listPeersIdsAndWatch

public java.util.List<java.lang.String> listPeersIdsAndWatch()

List this cluster's peers' IDs

Returns:

list of all peers' identifiers

getSlavesAddresses

public java.util.List<HServerAddress> getSlavesAddresses(java.lang.String peerClusterId)
                                                  throws org.apache.zookeeper.KeeperException

Returns all region servers from given peer

Parameters:

peerClusterId - (byte) the cluster to interrogate

Returns:

addresses of all region servers

Throws:

org.apache.zookeeper.KeeperException

connectToPeer

public boolean connectToPeer(java.lang.String peerId)
                      throws java.io.IOException,
                             org.apache.zookeeper.KeeperException

This method connects this cluster to another one and registers it in this region server's replication znode

Parameters:

peerId - id of the peer cluster

Throws:

org.apache.zookeeper.KeeperException

java.io.IOException

getPeer

public ReplicationPeer getPeer(java.lang.String peerId)
                        throws java.io.IOException,
                               org.apache.zookeeper.KeeperException

Helper method to connect to a peer

Parameters:

peerId - peer's identifier

Returns:

object representing the peer

Throws:

java.io.IOException

org.apache.zookeeper.KeeperException

setReplicating

public void setReplicating(boolean newState)
                    throws org.apache.zookeeper.KeeperException

Set the new replication state for this cluster

Parameters:

newState -

Throws:

org.apache.zookeeper.KeeperException

removePeer

public void removePeer(java.lang.String id)
                throws java.io.IOException

Remove the peer from zookeeper. which will trigger the watchers on every region server and close their sources

Parameters:

id -

Throws:

java.lang.IllegalArgumentException - Thrown when the peer doesn't exist

java.io.IOException

addPeer

public void addPeer(java.lang.String id,
                    java.lang.String clusterKey)
             throws java.io.IOException

Add a new peer to this cluster

Parameters:

id - peer's identifier

clusterKey - ZK ensemble's addresses, client port and root znode

Throws:

java.lang.IllegalArgumentException - Thrown when the peer doesn't exist

java.lang.IllegalStateException - Thrown when a peer already exists, since multi-slave isn't supported yet.

java.io.IOException

getReplication

public boolean getReplication()
                       throws org.apache.zookeeper.KeeperException

Get the replication status of this cluster. If the state znode doesn't exist it will also create it and set it true.

Returns:

returns true when it's enabled, else false

Throws:

org.apache.zookeeper.KeeperException

addLogToList

public void addLogToList(java.lang.String filename,
                         java.lang.String clusterId)

Add a new log to the list of hlogs in zookeeper

Parameters:

filename - name of the hlog's znode

clusterId - name of the cluster's znode

removeLogFromList

public void removeLogFromList(java.lang.String filename,
                              java.lang.String clusterId)

Remove a log from the list of hlogs in zookeeper

Parameters:

filename - name of the hlog's znode

clusterId - name of the cluster's znode

writeReplicationStatus

public void writeReplicationStatus(java.lang.String filename,
                                   java.lang.String clusterId,
                                   long position)

Set the current position of the specified cluster in the current hlog

Parameters:

filename - filename name of the hlog's znode

clusterId - clusterId name of the cluster's znode

position - the position in the file

Throws:

java.io.IOException

getRegisteredRegionServers

public java.util.List<java.lang.String> getRegisteredRegionServers()

Get a list of all the other region servers in this cluster and set a watch

Returns:

a list of server nanes

getListOfReplicators

public java.util.List<java.lang.String> getListOfReplicators()

Get the list of the replicators that have queues, they can be alive, dead or simply from a previous run

Returns:

a list of server names

getListPeersForRS

public java.util.List<java.lang.String> getListPeersForRS(java.lang.String rs)

Get the list of peer clusters for the specified server names

Parameters:

rs - server names of the rs

Returns:

a list of peer cluster

getListHLogsForPeerForRS

public java.util.List<java.lang.String> getListHLogsForPeerForRS(java.lang.String rs,
                                                                 java.lang.String id)

Get the list of hlogs for the specified region server and peer cluster

Parameters:

rs - server names of the rs

id - peer cluster

Returns:

a list of hlogs

lockOtherRS

public boolean lockOtherRS(java.lang.String znode)

Try to set a lock in another server's znode.

Parameters:

znode - the server names of the other server

Returns:

true if the lock was acquired, false in every other cases

copyQueuesFromRS

public java.util.SortedMap<java.lang.String,java.util.SortedSet<java.lang.String>> copyQueuesFromRS(java.lang.String znode)

This methods copies all the hlogs queues from another region server and returns them all sorted per peer cluster (appended with the dead server's znode)

Parameters:

znode - server names to copy

Returns:

all hlogs for all peers of that cluster, null if an error occurred

deleteSource

public void deleteSource(java.lang.String peerZnode,
                         boolean closeConnection)

Delete a complete queue of hlogs

Parameters:

peerZnode - znode of the peer cluster queue of hlogs to delete

deleteRsQueues

public void deleteRsQueues(java.lang.String znode)

Recursive deletion of all znodes in specified rs' znode

Parameters:

znode -

deleteOwnRSZNode

public void deleteOwnRSZNode()

Delete this cluster's queues

getHLogRepPosition

public long getHLogRepPosition(java.lang.String peerId,
                               java.lang.String hlog)
                        throws org.apache.zookeeper.KeeperException

Get the position of the specified hlog in the specified peer znode

Parameters:

peerId - znode of the peer cluster

hlog - name of the hlog

Returns:

the position in that hlog

Throws:

org.apache.zookeeper.KeeperException

registerRegionServerListener

public void registerRegionServerListener(ZooKeeperListener listener)

getClusterId

public java.lang.String getClusterId()

Get the identification of the cluster

Returns:

the id for the cluster

getPeerClusters

public java.util.Map<java.lang.String,ReplicationPeer> getPeerClusters()

Get a map of all peer clusters

Returns:

map of peer cluster keyed by id

getZNodeName

public static java.lang.String getZNodeName(java.lang.String fullPath)

Extracts the znode name of a peer cluster from a ZK path

Parameters:

fullPath - Path to extract the id from

Returns:

the id or an empty string if path is invalid

getZookeeperWatcher

public ZooKeeperWatcher getZookeeperWatcher()

Get this cluster's zk connection

Returns:

zk connection

getPeersZNode

public java.lang.String getPeersZNode()

Get the full path to the peers' znode

Returns:

path to peers in zk