Deploy your Bitnami Consul Stack on Microsoft Azure now! Launch Now

Bitnami Consul for Microsoft Azure

Description

Consul is a tool for discovering and configuring services in your infrastructure.

First steps with the Bitnami Consul Stack

Welcome to your new Bitnami application running on Microsoft Azure! Here are a few questions (and answers!) you might need when first starting with your application.

What credentials do I need?

You need a set of credentials: the server credentials that allow you to log in to your Microsoft Azure server using an SSH client and execute commands on the server using the command line. These credentials consist of an SSH username and key.

What SSH username should I use for secure shell access to my application?

SSH username: bitnami

How to start or stop the services?

Each Bitnami stack includes a control script that lets you easily stop, start and restart services. The script is located at /opt/bitnami/ctlscript.sh. Call it without any service name arguments to start all services:

$ sudo /opt/bitnami/ctlscript.sh start

Or use it to restart a single service, such as Apache only, by passing the service name as argument:

$ sudo /opt/bitnami/ctlscript.sh restart apache

Use this script to stop all services:

$ sudo /opt/bitnami/ctlscript.sh stop

Restart the services by running the script without any arguments:

$ sudo /opt/bitnami/ctlscript.sh restart

Obtain a list of available services and operations by running the script without any arguments:

$ sudo /opt/bitnami/ctlscript.sh

What are the default ports?

A port is an endpoint of communication in an operating system that identifies a specific process or a type of service. Bitnami stacks include several services or servers that require a port.

IMPORTANT: Making this application's network ports public is a significant security risk. You are strongly advised to only allow access to those ports from trusted networks. If, for development purposes, you need to access from outside of a trusted network, please do not allow access to those ports via a public IP address. Instead, use a secure channel such as a VPN or an SSH tunnel. Follow these instructions to remotely connect safely and reliably.

Port 22 is the default port for SSH connections.

The Consul access ports are 8300, 8301, 8302, 8500, 8600. These ports are closed by default. You must open them to enable remote access.

How to upload files to the server with SFTP?

NOTE: Bitnami applications can be found in /opt/bitnami/apps.

First, obtain your SSH credentials by following these steps:

  • Browse to the Bitnami Launchpad for Microsoft Azure and sign in if required using your Bitnami account.
  • Select the "Virtual Machines" menu item.
  • Select your cloud server from the resulting list.
  • Note the server IP address and SSH credentials on the resulting page. Your server may have been deployed using either an SSH password or an SSH key.

    SSH credentials with password

    SSH credentials with key

Although you can use any SFTP/SCP client to transfer files to your server, this guide documents FileZilla (Windows, Linux and Mac OS X), WinSCP (Windows) and Cyberduck (Mac OS X).

Using an SSH Key

Once you have your server's SSH key, choose your preferred application and follow the steps below to connect to the server using SFTP.

FileZilla
IMPORTANT: To use FileZilla, your server private key should be in PPK format.

Follow these steps:

  • Download and install FileZilla.
  • Launch FileZilla and use the "Edit -> Settings" command to bring up FileZilla's configuration settings.
  • Within the "Connection -> SFTP" section, use the "Add keyfile" command to select the private key file for the server. FileZilla will use this private key to log in to the server.

    FileZilla configuration

  • Use the "File -> Site Manager -> New Site" command to bring up the FileZilla Site Manager, where you can set up a connection to your server.
  • Enter your server host name and specify bitnami as the user name.
  • Select "SFTP" as the protocol and "Ask for password" as the logon type.

    FileZilla configuration

  • Use the "Connect" button to connect to the server and begin an SFTP session. You might need to accept the server key, by clicking "Yes" or "OK" to proceed.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

If you have problems accessing your server, get extra information by use the "Edit -> Settings -> Debug" menu to activate FileZilla's debug log.

FileZilla debug log

WinSCP
IMPORTANT: To use WinSCP, your server private key should be in PPK format.

Follow these steps:

  • Download and install WinSCP.
  • Launch WinSCP and in the "Session" panel, select "SCP" as the file protocol.
  • Enter your server host name and specify bitnami as the user name.

    WinSCP configuration

  • Click the "Advanced…" button and within the "SSH -> Authentication -> Authentication parameters" section, select the private key file for the server. WinSCP will use this private key to log in to the server.

    WinSCP configuration

  • From the "Session" panel, use the "Login" button to connect to the server and begin an SCP session.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

If you need to upload files to a location where the bitnami user doesn't have write permissions, you have two options:

  • Once you have configured WinSCP as described above, click the "Advanced…" button and within the "Environment -> Shell" panel, select sudo su - as your shell. This will allow you to upload files using the administrator account.

    WinSCP configuration

  • Upload the files to the /home/bitnami directory as usual. Then, connect via SSH and move the files to the desired location with the sudo command, as shown below:

     $ sudo mv /home/bitnami/uploaded-file /path/to/desired/location/
    
Cyberduck
IMPORTANT: To use Cyberduck, your server private key should be in PEM format.

Follow these steps:

  • Select the "Open Connection" command and specify "SFTP" as the connection protocol.

    Cyberduck configuration

  • In the connection details panel, under the "More Options" section, enable the "Use Public Key Authentication" option and specify the path to the private key file for the server.

    Cyberduck configuration

  • Use the "Connect" button to connect to the server and begin an SFTP session.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

Using a Password

Once you have your server's SSH credentials, choose your preferred application and follow the steps below to connect to the server using SFTP.

FileZilla

Follow these steps:

  • Download and install FileZilla.
  • Launch FileZilla and use the "File -> Site Manager -> New Site" command to bring up the FileZilla Site Manager, where you can set up a connection to your server.
  • Enter your server host name.
  • Select "SFTP" as the protocol and "Ask for password" as the logon type. Use bitnami as the server username and the password generated during the server deployment process.

    FileZilla configuration

  • Use the "Connect" button to connect to the server and begin an SFTP session. You might need to accept the server key, by clicking "Yes" or "OK" to proceed.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

If you have problems accessing your server, get extra information by use the "Edit -> Settings -> Debug" menu to activate FileZilla's debug log.

FileZilla debug log

WinSCP

Follow these steps:

  • Download and install WinSCP.
  • Launch WinSCP and in the "Session" panel, select "SCP" as the file protocol.
  • Enter your server host name and set bitnami as the server username. Enter the corresponding password as well.

    WinSCP configuration

  • From the "Session" panel, use the "Login" button to connect to the server and begin an SCP session.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

Cyberduck

Follow these steps:

  • Select the "Open Connection" command and specify "SFTP" as the connection protocol.

    Cyberduck configuration

  • In the connection details panel, enter the server IP address, bitnami as the username, and the password generated during the deployment process.

    Cyberduck configuration

  • Use the "Connect" button to connect to the server and begin an SFTP session.

You should now be logged into the /home/bitnami directory on the server. You can now transfer files by dragging and dropping them from the local server window to the remote server window.

How to create a Virtual Network peering?

To connect two instances internally you can enable a Virtual Network (VNet) peering from the Azure Portal. Depending if the instances were launched in the same or in different resource groups, there are two methods for performing a internal connection: sharing a virtual network or enabling a virtual network peering.

How to configure TLS authentication for Consul?

You can secure Consul by enabling TLS to verify the authenticity of servers and clients. This requires every key pair to be generated by a single Certificate Authority (CA). To enable TLS authentication, follow the instructions below:

Generate a private Certificate Authority (CA) certificate and key

IMPORTANT: To follow the steps below, you need to have the Go environment set up. Read the Go official documentation to learn how to install Go.
  • Install the CFSSL toolkit by running the commands below:

    $ go get -u github.com/cloudflare/cfssl/cmd/cfssl
    $ go get -u github.com/cloudflare/cfssl/cmd/cfssljson
    $ export PATH=$PATH:$HOME/go/bin
    
  • Generate a private CA certificate (consul-ca.pem) and key (consul-ca-key.pem):

    $ cfssl print-defaults csr > ca-csr.json && sed -i -e 's/256/2048/g' ca-csr.json && sed -i -e 's/ecdsa/rsa/g' ca-csr.json
    $ cfssl gencert -initca ca-csr.json | cfssljson -bare consul-ca
    

Generate certificates for your Consul servers and clients

The CA key is used to sign the certificates of each Consul node in your cluster. The CA certificate contains the public key used to validate the certificates and has to be distributed to every Consul node.

To generate and sign certificates for the Consul server and clients, follow these steps:

  • Create the cfssl.json configuration file below to increase the default certificate expiration time:

    $ sudo tee cfssl.json << 'EOF'
    {
      "signing": {
        "default": {
          "expiry": "87600h",
          "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
          ]
        }
      }
    }
    EOF
    
  • Consul certificates are signed by hostname (using the region and role) in the form ROLE.node.REGION.consul.
  • Generate a certificate for all the Consul servers in a specific region (global in this example):

    $ echo '{"key":{"algo":"rsa","size":2048}}' | cfssl gencert \
    -ca=consul-ca.pem -ca-key=consul-ca-key.pem -config=tls-ca/cfssl.json \
    -hostname="server.node.global.consul,localhost,127.0.0.1" - | \
    cfssljson -bare server
    
  • Generate a certificate for all the Consul clients in a specific region (global in this example):

    $ echo '{"key":{"algo":"rsa","size":2048}}' | cfssl gencert \
    -ca=consul-ca.pem -ca-key=consul-ca-key.pem -config=cfssl.json \
    -hostname="client.node.global.consul,localhost,127.0.0.1" - \
    | cfssljson -bare client
    
  • Generate a certificate for the CLI:

    $ echo '{"key":{"algo":"rsa","size":2048}}' | cfssl gencert \
    -ca=consul-ca.pem -ca-key=consul-ca-key.pem
    -profile=client - | cfssljson -bare cli
    

Configure your Consul servers and clients with the proper certificates

IMPORTANT: The steps below assumes you have installed the jq tool. It can be installed by running sudo apt-get install jq on Debian/Ubuntu or sudo yum install jq on CentOS.
  • Each Consul node should have the following keys and certificates:
    • Appropriate key file for its region and role (e.g. server-key.pem for the server)
    • Appropriate certificate file for its region and role (e.g. server.pem for the server)
    • CA's public certificate (consul-ca.pem)
  • Upload the following files to the /opt/bitnami/consul/certificates directory on each Consul server:
    • consul-ca.pem
    • server-key.pem
    • server.pem
    • cli-key.pem
    • cli.pem.

    Follow these instructions to upload files to the server with SFTP.

  • Upload the following files to each Consul client:

    • consul-ca.pem
    • client-key.pem
    • client.pem.

    Follow these instructions to upload files to the server with SFTP.

  • Configure your Consul server to verify incoming and outcoming connections. Connect to your server through SSH and run:

    $ tmp=$(mktemp) && jq '.verify_incoming = true' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
    $ tmp=$(mktemp) && jq '.verify_outgoing = true' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
    $ tmp=$(mktemp) && jq '.key_file = "/opt/bitnami/consul/certificates/server-key.pem"' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
    $ tmp=$(mktemp) && jq '.cert_file = "/opt/bitnami/consul/certificates/server.pem"' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
    $ tmp=$(mktemp) && jq '.ca_file = "/opt/bitnami/consul/certificates/consul-ca.pem"' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
    $ tmp=$(mktemp) && jq '.ports.https = 8443' /opt/bitnami/consul/conf/consul.json > "${tmp}" && sudo mv "${tmp}" /opt/bitnami/consul/conf/consul.json
    
  • Restart Consul and check you can connect through HTTPS using the CLI by running:

    $ sudo /opt/bitnami/ctlscript.sh restart consul
    
    $ consul members -ca-file=/opt/bitnami/consul/certificates/consul-ca.pem -client-cert=/opt/bitnami/consul/certificates/cli.pem -client-key=/opt/bitnami/consul/certificates/cli-key.pem -http-addr="https://localhost:8443"
    

How to connect to Consul from a different machine?

For security reasons, the Consul ports in this solution cannot be accessed over a public IP address. To connect to Consul from a different machine, you must open port Bitnami does not open any port by default in this solution for remote access. Refer to the FAQ for more information on this.

IMPORTANT: Making this application's network ports public is a significant security risk. You are strongly advised to only allow access to those ports from trusted networks. If, for development purposes, you need to access from outside of a trusted network, please do not allow access to those ports via a public IP address. Instead, use a secure channel such as a VPN or an SSH tunnel. Follow these instructions to remotely connect safely and reliably.

How to create a Consul cluster?

This section describes the creation of a Consul cluster with servers located on different hosts. Follow the instructions below to create a cluster comprised of three instances.

To begin:

  • Launch as many Consul instances as you need for the cluster (in this example, three instances).
  • Connect them using VPC Network Peering.

Configure the first Consul instance

The first instance does not require any special configuration.

Configure the other Consul instances

On each of the other instances, perform the steps below:

  • Edit the /opt/bitnami/consul/conf/consul.json file and set the following parameter:

      "bootstrap_expect":0,
    
  • Restart the Consul service:

      $ sudo /opt/bitnami/ctlscript.sh restart consul
    
  • Run the following command to join to the cluster. Remember to replace IP_ADDRESS_SERVER1 with the internal IP address of the first Consul instance.

      $ consul join IP_ADDRESS_SERVER1
    

Your Consul cluster is now operational. To test it, see the steps in the following section.

Test the cluster

To view the members of the cluster, execute the following commands:

$ consul members
$ consul operator raft list-peers

You should see output similar to the below:

$ consul members
Node          Address          Status  Type    Build  Protocol  DC   Segment
7a09a3f508af  XX.XX.XX.XX:8301  alive   server  1.2.0  2         dc1  <all>
ca24bba7fe91  XX.XX.XX.XX:8301  alive   server  1.2.0  2         dc1  <all>
ee418517cbb5  XX.XX.XX.XX:8301  alive   server  1.2.0  2         dc1  <all>

$ consul operator raft list-peers
Node          ID                                    Address          State     Voter  RaftProtocol
ca24bba7fe91  04f8464f-ae77-af99-6d76-829928f67e82  XX.XX.XX.XX:8300  leader    true   3
ee418517cbb5  484d857e-894f-4351-ee99-b6d51aa7e481  XX.XX.XX.XX:8300  follower  true   3
7a09a3f508af  5a7b8c4a-d074-1051-56d4-d0e442e713c6  XX.XX.XX.XX:8300  follower  true   3

To test data replication, follow these steps:

  • On the first Consul instance, create a key-value pair. The example below create a key named example/data with value test.

      $ consul kv put example/data test
    
  • On any other Consul instance, retrieve the value of the key by executing the following:

      $ consul kv get example/data
    

If you see the value test, this indicates that data is successfully replicating across the cluster.

How can I run a command in the Bitnami Consul Stack?

Log in to the server console as the bitnami user and run the command as usual. The required environment is automatically loaded for the bitnami user.

How to create a full backup of Consul?

Backup

The Bitnami Consul Stack is self-contained and the simplest option for performing a backup is to copy or compress the Bitnami stack installation directory. To do so in a safe manner, you will need to stop all servers, so this method may not be appropriate if you have people accessing the application continuously.

Follow these steps:

  • Change to the directory in which you wish to save your backup:

      $ cd /your/directory
    
  • Stop all servers:

      $ sudo /opt/bitnami/ctlscript.sh stop
    
  • Create a compressed file with the stack contents:

      $ sudo tar -pczvf application-backup.tar.gz /opt/bitnami
    
  • Restart all servers:

      $ sudo /opt/bitnami/ctlscript.sh start
    

You should now download or transfer the application-backup.tar.gz file to a safe location.

Restore

Follow these steps:

  • Change to the directory containing your backup:

      $ cd /your/directory
    
  • Stop all servers:

      $ sudo /opt/bitnami/ctlscript.sh stop
    
  • Move the current stack to a different location:

      $ sudo mv /opt/bitnami /tmp/bitnami-backup
    
  • Uncompress the backup file to the original directoryv

      $ sudo tar -pxzvf application-backup.tar.gz -C /
    
  • Start all servers:

      $ sudo /opt/bitnami/ctlscript.sh start
    

If you want to create only a database backup, refer to these instructions for MySQL and PostgreSQL.

azure

Bitnami Documentation