Adding / Removing Nodes

This page describes the process of adding and removing nodes to and from a Riak KV cluster. For information on creating a cluster check out Running a Cluster.

Start the Node

Just like the initial configuration steps, this step has to be repeated for every node in your cluster. Before a node can join an existing cluster it needs to be started. Depending on your mode of installation, use either the init scripts installed by the Riak binary packages or simply the script riak:

/etc/init.d/riak start

or

bin/riak start

When the node starts, it will look for a cluster description, known as the ring file, in its data directory. If a ring file does not exist, it will create a new ring file based on the initially configured ring_size (or ring_creation_size if you’re using the older, app.config-based configuration system), claiming all partitions for itself. Once this process completes, the node will be ready to serve requests.

Add a Node to an Existing Cluster

Once the node is running, it can be added to an existing cluster. Note that this step isn’t necessary for the first node; it’s necessary only for nodes that you want to add later.

To join the node to an existing cluster, use the cluster join command:

bin/riak admin cluster join <node_in_cluster>

The <node_in_cluster> in the example above can be any node in the cluster you want to join to. So if the existing cluster consists of nodes A, B, and C, any of the following commands would join the new node:

bin/riak admin cluster join A
bin/riak admin cluster join B
bin/riak admin cluster join C

To give a more realistic example, let’s say that you have an isolated node named riak@192.168.2.5 and you want to join it to an existing cluster that contains a node named riak@192.168.2.2. This command would stage a join to that cluster:

bin/riak admin cluster join riak@192.168.2.2

If the join request is successful, you should see the following:

Success: staged join request for 'riak@192.168.2.5' to 'riak@192.168.2.2'

If you have multiple nodes that you would like to join to an existing cluster, repeat this process for each of them.

Joining Nodes to Form a Cluster

The process of joining a cluster involves several steps, including staging the proposed cluster nodes, reviewing the cluster plan, and committing the changes.

After staging each of the cluster nodes with riak admin cluster join commands, as in the section above, the next step in forming a cluster is to review the proposed plan of changes. This can be done with the riak admin cluster plan command, which is shown in the example below.

=============================== Staged Changes ================================
Action         Nodes(s)
-------------------------------------------------------------------------------
join           'riak@192.168.2.2'
join           'riak@192.168.2.2'
join           'riak@192.168.2.2'
join           'riak@192.168.2.2'
-------------------------------------------------------------------------------

NOTE: Applying these changes will result in 1 cluster transition

###############################################################################
                         After cluster transition 1/1
###############################################################################

================================= Membership ==================================
Status     Ring    Pending    Node
-------------------------------------------------------------------------------
valid     100.0%     20.3%    'riak@192.168.2.2'
valid       0.0%     20.3%    'riak@192.168.2.3'
valid       0.0%     20.3%    'riak@192.168.2.4'
valid       0.0%     20.3%    'riak@192.168.2.5'
valid       0.0%     18.8%    'riak@192.168.2.6'
-------------------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Transfers resulting from cluster changes: 51
  12 transfers from 'riak@192.168.2.2' to 'riak@192.168.2.3'
  13 transfers from 'riak@192.168.2.2' to 'riak@192.168.2.4'
  13 transfers from 'riak@192.168.2.2' to 'riak@192.168.2.5'
  13 transfers from 'riak@192.168.2.2' to 'riak@192.168.2.6'

If the plan is to your liking, submit the changes by running riak admin cluster commit.

Removing a Node From a Cluster

A node can be removed from the cluster in two ways. One assumes that a node is decommissioned, for example, because its added capacity is not needed anymore or because it’s explicitly replaced with a new one. The second is relevant for failure scenarios in which a node has crashed and is irrecoverable and thus must be removed from the cluster from another node.

The command to remove a running node is riak admin cluster leave. This command must be executed on the node that you intend to removed from the cluster.

Similarly to joining a node, after executing riak admin cluster leave the cluster plan must be reviewed with riak admin cluster plan and the changes committed with riak admin cluster commit.

The other command is riak admin cluster leave <node>, where <node> is the node name as specified in the node’s configuration files:

riak admin cluster leave riak@192.168.2.1

This command can be run from any other node in the cluster.

Under the hood, both commands do basically the same thing. Running riak admin cluster leave <node> selects the current node for you automatically.

As with riak admin cluster leave, the plan to have a node leave the cluster must be first reviewed with riak admin cluster plan and committed with riak admin cluster commit before any changes will actually take place.

Pausing a join or leave

To pause during riak admin cluster join or riak admin cluster leave, set the node’s transfer-limit to 0:

riak admin transfer-limit <node> 0