googleapache-airflow

Create a Redis cluster

From Redis 3.x, it is possible to create a cluster of Redis nodes in primary-replica model. The minimum configuration required is a 6-node cluster with 3 primary and 3 replicas. This section will introduce you to creating a simple redis cluster. For further information, please refer to http://redis.io/topics/cluster-tutorial and http://redis.io/topics/cluster-spec.

Pre-requisites

Ruby must be installed with the Redis gem. Install these components with the commands below:

    $ sudo apt-get install ruby
    $ sudo gem install redis

Manual approach

Use the following template and place it in your desired location.

  • Create the following folder structure:

      $ mkdir -p my-cluster
  $ cd my-cluster
  $ mkdir PORT-1 PORT-2 PORT-3 PORT-4 PORT-5 PORT-6

  • In every PORT-* folder, create a configuration file. Name it whatever you wish (eg. *PORT-x.conf*) and fill it with this content:

      port PORT-x
  cluster-enabled yes
  cluster-config-file nodes-PORT-x.conf
  cluster-node-timeout 5000
  appendonly yes

  • Start a Redis server (node) in each port. This will open the normal port for client connection and the cluster-bus port (10000 + PORT-x), used between nodes to communicate.

      $ cd PORT-x
  $ ../redis-server ./PORT-x.conf

  • Execute the cluster manager (redis-trib.rb) located in the /opt/bitnami/redis/bin directory:

      $ ./redis-trib.rb create --replicas 1 127.0.0.1:PORT1 127.0.0.1:PORT-2 127.0.0.1:PORT-3 127.0.0.1:PORT-4 127.0.0.1:PORT-5 127.0.0.1:PORT-6

  • Connect to each node in the cluster as usual.

Automatic approach

Redis comes with a tool named create-cluster, located at /opt/bitnami/scripts/create-cluster. This allows you to avoid the manual configuration described above.

By default, this utility will create 6 nodes with 1 replica and will start creating nodes on port 30000. In order to not modify the utility, is recommended to create a config.sh script in the same folder as create-cluster with the following content:

PORT=STARTING-PORT-NUMBER
TIMEOUT=2000
NODES=6
REPLICAS=1

Start the node and create the cluster:

$ ./create-cluster start
$ ./create-cluster create

Stop the cluster:

$ ./create-cluster stop

Clean up the folder:

$ ./create-cluster clean
Last modification December 21, 2022