Backup and Restore MariaDB Galera Deployments on Kubernetes

Introduction

MariaDB Galera Cluster makes it easy to create an active-active multi-master cluster while still retaining all the familiar MariaDB clients and tools. Bitnami offers a MariaDB Galera Helm chart that makes it quick and easy to deploy such a cluster on Kubernetes. This Helm chart is compliant with current best practices and can also be easily upgraded to ensure that you always have the latest fixes and security updates.

Once you have a MariaDB Galera Cluster deployed, you need to start thinking about ongoing maintenance and disaster recovery, and begin putting a data backup/restore strategy in place. This backup/restore strategy is needed for many operational scenarios, including disaster recovery planning, off-site data analysis or application load testing.

This guide walks you through the process of backing up and restoring MariaDB Galera Cluster deployments on Kubernetes using MariaDB's built-in backup/restore tools.

Assumptions and prerequisites

This guide makes the following assumptions:

  • You have two separate Kubernetes clusters - a source cluster and a destination cluster - with kubectl and Helm v3 installed. This guide uses Google Kubernetes Engine (GKE) clusters but you can also use any other Kubernetes provider. Learn about deploying a Kubernetes cluster on different cloud platforms and how to install kubectl and Helm.

  • You have previously deployed the Bitnami MariaDB Galera Helm chart on the source cluster and added some data to it. Example command sequences to perform these tasks are shown below, where the PASSWORD and REPL-PASSWORD placeholders refer to the database administrator and replication user passwords respectively.

    helm repo add bitnami https://charts.bitnami.com/bitnami
    helm install galera bitnami/mariadb-galera \
      --namespace default \
      --set rootUser.password=PASSWORD \
      --set galera.mariabackup.password=REPL-PASSWORD
    kubectl run galera-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.4.12-debian-10-r78 --command -- mysql -h galera-mariadb-galera -P 3306 -uroot -pPASSWORD
    CREATE DATABASE mydb;
    USE mydb;
    CREATE TABLE accounts (name VARCHAR(255) NOT NULL, total INT NOT NULL);
    INSERT INTO accounts VALUES ('user1', '647'), ('user2', '573');
    exit
    

The method described below involves using MariaDB's mysqldump tool to create a point-in-time backup of the data in the source cluster, and then using the mysql tool to restore this data on the destination cluster.

Step 1: Backup data with mysqldump

The first step is to back up the data in the MariaDB Galera source cluster. Follow these steps:

  • Obtain the MariaDB Galera Cluster's administrator password:

    export PASSWORD=$(kubectl get secret --namespace default galera-mariadb-galera -o jsonpath="{.data.mariadb-root-password}" | base64 --decode)
    
  • Forward the MariaDB Galera Cluster service port and place the process in the background:

    kubectl port-forward --namespace default svc/galera-mariadb-galera 3306:3306 &
    
  • Create a directory for the backup files and make it the current working directory:

    mkdir mybackup
    cd mybackup
    
  • Back up the contents of all the databases to the current directory using the mysqldump tool. If this tool is not installed on your system, use Bitnami's MariaDB Galera Docker image to perform the backup, as shown below. Replace the PASSWORD placeholder with the database administrator password.

    docker run --rm --name mysqldump -v $(pwd):/app --net="host" bitnami/mariadb-galera:latest mysqldump -h 127.0.0.1 -u root -pPASSWORD -A > backup.sql
    

    Here, the --net parameter lets the Docker container use the host's network stack and thereby gain access to the forwarded port. The mysqldump command connects to the forwarded MariaDB Galera service and creates an SQL backup file in the /app directory, which is mapped to the current directory (mybackup/) on the Docker host. Finally, the --rm parameter deletes the container after the mysqldump command completes execution.

  • Stop the service port forwarding by terminating the corresponding background process.

At the end of this step, the backup directory should contain a file with the data from your running MariaDB Galera Cluster deployment.

Step 2: Restore data with mysql

The next step is to create an empty MariaDB Galera Cluster deployment on the destination cluster and restore the data into it. You can also use the procedure shown below with a new MariaDB Galera Cluster deployment in a separate namespace in the same cluster.

  • Create a new MariaDB Galera Cluster deployment. Replace the PASSWORD and REPL-PASSWORD placeholders with the database administrator and replication user passwords.

    helm repo add bitnami https://charts.bitnami.com/bitnami
    helm install galera-new bitnami/mariadb-galera \
        --set rootUser.password=PASSWORD \
        --set galera.mariabackup.password=REPL-PASSWORD
    
  • Forward the MariaDB Galera Cluster service port for the new deployment and place the process in the background:

    kubectl port-forward --namespace default svc/galera-new-mariadb-galera 3306:3306 &
    
  • Create and start a Bitnami MariaDB Galera container image. Mount the directory containing the backup file as a volume in this container and use the mysql client tool in the container image to import the backup into the new cluster, as shown below. Replace the PASSWORD placeholder with the database administrator password.

    cd mybackup
    docker create -t --rm --name galera-client -v $(pwd):/app --net="host" bitnami/mariadb-galera:latest bash
    docker start galera-client
    docker exec -i galera-client mysql -h 127.0.0.1 -uroot -pPASSWORD < backup.sql
    

    Here, the -v parameter mounts the current directory (containing the backup file) to the container's /app path. Then, the mysql client tool is used to connect to the new MariaDB Galera Cluster service and restore the data from the original deployment. The --rm parameter destroys the container on exit.

  • Stop the service port forwarding by terminating the background process.

  • Connect to the new deployment and confirm that your data has been successfully restored:

    kubectl run galera-new-mariadb-galera-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/mariadb-galera:10.4.12-debian-10-r78 --command -- mysql -h galera-new-mariadb-galera -P 3306 -uroot -pPASSWORD -e "SELECT * FROM mydb.accounts"
    

    Here is an example of what you should see:

Query results

Useful links