Bitnami ZooKeeper Virtual Machine

IMPORTANT: The ZooKeeper OVA has been released for VMware vSphere only. The following information does not apply for VirtualBox or similar VM executors.

Description

ZooKeeper provides a reliable, centralized register of configuration data and services for distributed applications.

First steps with the Bitnami ZooKeeper Stack

Welcome to your new Bitnami application! This guide includes some basic information you will need to get started with your application.

What credentials do I need?

You need two sets of credentials:

  • The application credentials, consisting of a username and password. These credentials allow you to log in to your new Bitnami application.
  • The server credentials, consisting of an SSH username and SSH key file. These credentials allow you to log in to your virtual machine using an SSH client and execute commands on the server using the command line. Contact your datacenter administrator for more information.

What is the administrator username set for me to log in to the application for the first time?

Username: zkcli

What is the administrator password?

Password: The administrator password to log in to your application is randomly generated during the first boot. Check the FAQ to learn how to retrieve it.

What is my server IP address?

The IP address is displayed on screen at the end of the boot process, but you can check it at any time by running the following command:

$ sudo ifconfig

Check server IP address

How to start or stop the services?

Each Bitnami stack includes a control script that lets you easily stop, start and restart services. The script is located at /opt/bitnami/ctlscript.sh. Call it without any service name arguments to start all services:

$ sudo /opt/bitnami/ctlscript.sh start

Or use it to restart a single service, such as Apache only, by passing the service name as argument:

$ sudo /opt/bitnami/ctlscript.sh restart apache

Use this script to stop all services:

$ sudo /opt/bitnami/ctlscript.sh stop

Restart the services by running the script without any arguments:

$ sudo /opt/bitnami/ctlscript.sh restart

Obtain a list of available services and operations by running the script without any arguments:

$ sudo /opt/bitnami/ctlscript.sh

ZooKeeper default configuration

ZooKeeper configuration files

The ZooKeeper configuration files are located at the /opt/bitnami/zookeeper/conf/ directory.

ZooKeeper ports

By default, the ZooKeeper server runs on port 2181. Only conections from localhost are permitted.

ZooKeeper log files

The ZooKeeper log file is /opt/bitnami/zookeeper/logs/zookeeper.out.

What are the default ports?

A port is an endpoint of communication in an operating system that identifies a specific process or a type of service. Bitnami stacks include several services or servers that require a port.

IMPORTANT: Making this application's network ports public is a significant security risk. You are strongly advised to only allow access to those ports from trusted networks. If, for development purposes, you need to access from outside of a trusted network, please do not allow access to those ports via a public IP address. Instead, use a secure channel such as a VPN or an SSH tunnel. Follow these instructions to remotely connect safely and reliably.

Port 22 is the default port for SSH connections.

The ZooKeeper access port is 2181. This port is closed by default. You must open it to enable remote access.

How to connect to ZooKeeper?

You can connect to ZooKeeper from the same server where it is installed with the ZooKeeper client tool. Execute the following command:

$ zkCli.sh

You should see an output similar to this:

Connect to ZooKeeper

You can run the following example to see how to add new data to ZooKeeper, recover it, and delete the created registries:

  • Use the list command to see the data stored in the instance:

    $ ls /
    [zookeeper]
    
  • Create a new directory (in this example, test directory which includes the string mydata) using the create / command. This creates a new znode and associates the string "my_data" with the node. You should see:

    $ create /test mydata
    Created /test
    
  • Use the list command again to see how the directory looks like now:

    $ ls /
    [zookeeper, test]
    
  • Check the information associated to the new node by using the get / command:

    $ get /test
    mydata
    cZxid = 0x23
    ctime = Fri Jun 01 14:43:23 UTC 2018
    mZxid = 0x23
    mtime = Fri Jun 01 14:43:23 UTC 2018
    pZxid = 0x23
    cversion = 0
    dataVersion = 0
    aclVersion = 0
    ephemeralOwner = 0x0
    dataLength = 6
    numChildren = 0
    
  • Change the data associated to the test node using the set / command:

    $ set /test apache
    cZxid = 0x23
    ctime = Fri Jun 01 14:43:23 UTC 2018
    mZxid = 0x24
    mtime = Fri Jun 01 14:51:07 UTC 2018
    pZxid = 0x23
    cversion = 0
    dataVersion = 1
    aclVersion = 0
    ephemeralOwner = 0x0
    dataLength = 6
    numChildren = 0
    

    Run again the get / command to check that the data has been changed from mydata to apache:

    $ get /test
    apache
    cZxid = 0x23
    ctime = Fri Jun 01 14:43:23 UTC 2018
    mZxid = 0x24
    mtime = Fri Jun 01 14:51:07 UTC 2018
    pZxid = 0x23
    cversion = 0
    dataVersion = 1
    aclVersion = 0
    ephemeralOwner = 0x0
    dataLength = 6
    numChildren = 0
    
  • Delete the node with the delete / command:

    $ delete /test
    

    If you execute again the ls / command you should see only the [zookeeper] node.

How to connect to ZooKeeper from a different machine?

The ZooKeeper OVA can only be deployed within a VMware vSphere environment. For that reason, it does not include any specialized firewall software pre-installed or firewall rules set. It is assumed that this OVA will run on a private LAN.

Please contact your system administrator to learn how to remotely access this machine.

How to create a ZooKeeper ensemble?

This section describes the creation of a ZooKeeper ensemble with servers located on different hosts. The following example shows an ensemble comprised of three instances, follow these instructions:

  • Launch as much ZooKeeper instances as nodes you want to have in the ensemble. (In this example, three instances).

Configuring the first ZooKeeper instance

  • Check that the value of the /opt/bitnami/zookeeper/data/myid file is equal to 1.
  • Edit the /opt/bitnami/zookeeper/conf/zoo.cfg file to add the following content. Remember to replace IP_ADDRESS with the corresponding IP address of each instance.

    server.1=IP_ADDRESS_SERVER1:2888:3888
    server.2=IP_ADDRESS_SERVER2:2888:3888
    server.3=IP_ADDRESS_SERVER3:2888:3888
    
  • Restart ZooKeeper services:

    $ sudo /opt/bitnami/ctlscript.sh restart zookeeper
    

Configuring the second ZooKeeper instance

  • Edit the /opt/bitnami/zookeeper/data/myid file and change the value 1 to 2.
  • Edit the /opt/bitnami/zookeeper/conf/zoo.cfg file to add the following content. Remember to replace IP_ADDRESS with the corresponding IP address of each instance.

    server.1=IP_ADDRESS_SERVER1:2888:3888
    server.2=IP_ADDRESS_SERVER2:2888:3888
    server.3=IP_ADDRESS_SERVER3:2888:3888
    
  • Restart ZooKeeper services:

    $ sudo /opt/bitnami/ctlscript.sh restart zookeeper
    

Configuring the third ZooKeeper instance

  • Edit the /opt/bitnami/zookeeper/data/myid file and change the value 1 to 3.
  • Edit the /opt/bitnami/zookeeper/conf/zoo.cfg file to add the following content. Remember to replace IP_ADDRESS with the corresponding IP address of each instance.

    server.1=IP_ADDRESS_SERVER1:2888:3888
    server.2=IP_ADDRESS_SERVER2:2888:3888
    server.3=IP_ADDRESS_SERVER3:2888:3888
    
  • Restart ZooKeeper services:

    $ sudo /opt/bitnami/ctlscript.sh restart zookeeper
    

Testing the ensemble

To check if the replication is working correctly in the cluster, let's try the following:

  • In the server 1, connect to ZooKeeper by executing the following:

    $ zkCli.sh
    
  • Create a new directory named ensemble with a string named test using the create / command. This creates a new znode and associates the string test with the node. You should see:

    $ create /ensemble test
    Created /ensemble
    
  • Use the list command again to see how the directory looks like now:

    $ ls /
    [zookeeper, ensemble]
    
  • Go to the server 2 and connect to ZooKeeper using the zkCli.sh command.
  • Use the ls / command to check that you have the same data in both instances. You should see the same output you have obtained in the server 1 when executing that command:

    $ ls /
    [zookeeper, ensemble]
    

Congratulations! You now have a ZooKeeper ensemble running and ready to replicate the data through all the instances/nodes.

How can I run a command in the Bitnami ZooKeeper Stack?

Log in to the server console as the bitnami user and run the command as usual. The required environment is automatically loaded for the bitnami user.

How to start/stop the ZooKeeper server?

  • To start the Zookeeper server, access your machine and execute the following:

    $ sudo /opt/bitnami/ctlscript.sh start zookeeper
    
  • To stop the Zookeeper server, execute the following:

    $ sudo /opt/bitnami/ctlscript.sh stop zookeeper
    

How to create a full backup of ZooKeeper?

Backup

The Bitnami ZooKeeper Stack is self-contained and the simplest option for performing a backup is to copy or compress the Bitnami stack installation directory. To do so in a safe manner, you will need to stop all servers, so this method may not be appropriate if you have people accessing the application continuously.

Follow these steps:

  • Change to the directory in which you wish to save your backup:

      $ cd /your/directory
    
  • Stop all servers:

      $ sudo /opt/bitnami/ctlscript.sh stop
    
  • Create a compressed file with the stack contents:

      $ sudo tar -pczvf application-backup.tar.gz /opt/bitnami
    
  • Restart all servers:

      $ sudo /opt/bitnami/ctlscript.sh start
    

You should now download or transfer the application-backup.tar.gz file to a safe location.

Restore

Follow these steps:

  • Change to the directory containing your backup:

      $ cd /your/directory
    
  • Stop all servers:

      $ sudo /opt/bitnami/ctlscript.sh stop
    
  • Move the current stack to a different location:

      $ sudo mv /opt/bitnami /tmp/bitnami-backup
    
  • Uncompress the backup file to the original directoryv

      $ sudo tar -pxzvf application-backup.tar.gz -C /
    
  • Start all servers:

      $ sudo /opt/bitnami/ctlscript.sh start
    

If you want to create only a database backup, refer to these instructions for MySQL and PostgreSQL.

How to debug Zookeeper errors?

The main Zookeeper log file is created at /opt/bitnami/zookeeper/logs/zookeeper.out.

virtualMachine

Bitnami Documentation