Monitor a MariaDB Replication Cluster on Kubernetes with Prometheus and Grafana


MariaDB is an open source relational database management system that is widely used for enterprise applications and mission-critical use cases. One of its key features is data replication, which allows data to be mirrored across multiple nodes and thereby avoid critical data loss. To make it easy to deploy MariaDB in production Kubernetes environments, Bitnami offers a MariaDB Helm chart. The Bitnami MariaDB chart sets up a master-slave MariaDB streaming replication cluster on Kubernetes that is easily configurable, fault-tolerant and follows best practices for security and scalability.

However, for DBAs and administrators, setting up a MariaDB replication cluster on Kubernetes is only part of the challenge. Equally important for them is the ability to monitor the cluster on an ongoing basis so that they can proactively respond to cluster health events.

This guide covers the process of installing Prometheus and Grafana on Kubernetes and deploying a MariaDB replication cluster with Prometheus metrics enabled. This gives DBAs and system administrators a way to visually monitor MariaDB cluster health and replication activity, obtain up-to-the-minute metrics and gain better insight into cluster usage.

Assumptions and prerequisites

This guide makes the following assumptions:

Step 1: Install Prometheus

The first step is to install the Prometheus Operator in the cluster, as follows:

helm repo add bitnami
helm repo update
helm install prometheus bitnami/prometheus-operator

Once the deployment is complete, note the DNS name for the service within the cluster. You will need this name when configuring Grafana in a later step.

Prometheus service name

If you wish to log in to the Prometheus dashboard, follow the post-deployment instructions to proxy HTTP connections using kubectl.

Step 2: Install Grafana

The next step is to install Grafana using Helm. There are various options available here, depending on your access requirements.

Access Grafana through the Kubernetes proxy

You can install Grafana such that the service is only available through the Kubernetes proxy. This is the most secure option, as it precludes any external access. Use the command below, replacing the GRAFANA-PASSWORD placeholder with an actual password:

helm install grafana bitnami/grafana \
    --set admin.password=GRAFANA-PASSWORD

Configure kubectl to forward requests to Grafana using the commands below:

kubectl port-forward svc/grafana2 8080:3000 &

Once the deployment is complete, you should be able to access Grafana by browsing to http://localhost:8080 on the host system and logging in with the username admin and the configured password.

Access Grafana over HTTP using a domain name


If you wish to access Grafana externally through a domain name, you must first install the NGINX Ingress controller.

To configure Grafana to be accessible externally through a domain name, use the command below. Replace the GRAFANA-PASSWORD placeholder with an actual password and the DOMAIN placeholder with the domain name you wish to use.

helm install grafana bitnami/grafana \
    --set admin.password=GRAFANA-PASSWORD \
    --set ingress.enabled=true \
    --set ingress.hosts[0].name={DOMAIN}

Once the deployment is complete, you should be able to access Grafana by browsing to http://DOMAIN and logging in with the username admin and the configured password.

If you wish to access Grafana over HTTPS, you must install and configure cert-manager for certificate generation and then upgrade your Grafana deployment using the necessary parameters for TLS. See an example of this in our Ingress with TLS guide.

Here is what you should see after logging in:

Grafana dashboard

Step 3: Create a Prometheus data source in Grafana

Follow these steps:

  • From the Grafana dashboard, click the "Add data source" button.
  • On the "Choose data source type" page, select "Prometheus".
Grafana data source
  • On the "Settings" page, set the URL for the Prometheus server to http://NAME:9090, where NAME is the DNS name for the Prometheus service obtained at the end of Step 1. Leave all other values at their default.
Grafana data source configuration
  • Click "Save & Test" to save and test the configuration. If everything is configured correctly, you should see a success message like the one below.
Grafana test
  • Click the "Dashboards" tab and import the available Prometheus dashboards (optional).
Grafana Prometheus dashboards

Step 4: Deploy the MariaDB replication cluster

Once Prometheus and Grafana are configured, you can proceed to deploy the MariaDB replication cluster using the Bitnami MariaDB Helm chart. With the chart, Bitnami provides two configuration files: values.yaml, which initializes the deployment using a set of default values and is intended for development or test environments, and values-production.yaml, which is intended for production environments.

Deploy the MariaDB cluster with one master and two slaves using the production configuration using the command below. Replace the MARIADB-ADMIN-PASSWORD and MARIADB-REPL-PASSWORD placeholders with the database administrator account and replication account password respectively. You can also optionally create a MariaDB user account for application use by specifying values for the USER-PASSWORD, USER-NAME and DB-NAME placeholders.

helm install mariadb bitnami/mariadb -f values-production.yaml \
    --set rootUser.password=MARIADB-ADMIN-PASSWORD \
    --set replication.password=MARIADB-REPL-PASSWORD \
    --set db.user=USER-NAME \
    --set db.password=USER-PASSWORD \
    --set \
    --set slave.replicas=1 \
    --set metrics.enabled=true \
    --set metrics.serviceMonitor.enabled=true

Note the metrics.enabled and metrics.serviceMonitor.enabled parameters, which enable Prometheus export for server metrics.

Once the deployment is complete, note the instructions to connect to the database service and proceed to the next step.

Step 5: Add MariaDB dashboards to Grafana

You can now proceed to add dashboards to Grafana for MariaDB monitoring. A number of community-created dashboards are available on the Grafana website and in the Percona GitHub repository.

Here is an example of adding the Percona MySQL Overview dashboard to Grafana:

Grafana dashboard import
  • Click "Load" to load the data and then "Import" to import the dashboard.

The new dashboard should appear in Grafana.

To test the dashboard, connect to the MariaDB service using the instructions provided at the time of deploying the Helm chart and run a few example queries. You should see your activity reflected in the MySQL Overview chart, as in the example below:

Grafana MySQL dashboard

You now have a MariaDB cluster running on Kubernetes with Prometheus metrics enabled and available through Grafana.

Appendix: Scale the deployment

You can easily scale the cluster up or down by adding or removing nodes. For example, to scale the number of slaves up to 5, get the name of the slave StatefulSet and then use kubectl, as shown below:

kubectl get sts -l "release=mariadb,component=slave"
kubectl scale sts/mariadb-slave --replicas=5

Useful links

To learn more about the topics discussed in this guide, use the links below: