generaledx

Migrate Open edX

NOTE: We are in the process of modifying the file structure and configuration for many Bitnami stacks. On account of these changes, the file paths stated in this guide may change depending on whether your Bitnami stack uses native Linux system packages (Approach A), or if it is a self-contained installation (Approach B). To identify your Bitnami installation type and what approach to follow, run the command below:

 $ test ! -f "/opt/bitnami/common/bin/openssl" && echo "Approach A: Using system packages." || echo "Approach B: Self-contained installation."

The output of the command indicates which approach (A or B) is used by the installation, and will allow you to identify the paths, configuration and commands to use in this guide. Refer to the FAQ for more information on these changes.

IMPORTANT: Before proceeding to migrate the Bitnami edX stack, ensure that you have performed a complete backup.

Migrate edX

Backup data from the old instance

Log in to your edX instance and follow the steps below:

  • Create a directory where to store all the data:

      $ mkdir edx-migrate-data

  • Dump databases required by edX:

    • MySQL:

        $ mysqldump -u root -p --databases bitnami_edxapp bitnami_edxapp_csmh bitnami_xqueue --add-drop-database > edx-migrate-data/bitnami_edx_mysql.sql

    • MongoDB:

        $ mongodump --authenticationDatabase admin -h localhost -u root -d bitnami_edxapp --out edx-migrate-data/bitnami_edx_mongodb
  $ mongodump --authenticationDatabase admin -h localhost -u root -d bitnami_cs_comments_service --out edx-migrate-data/bitnami_edx_mongodb

  • Copy edX Platform configuration and data files (which contain uploads, courses, etc.). Depending on your installation type, execute the following commands:

    NOTE: The commands below also clear the assets to reduce the size of the directory:

    • Approach A (Bitnami installations using system packages):

        $ cp /opt/bitnami/edx/etc/*.yml edx-migrate-data
  $ cp -R /opt/bitnami/edx/var edx-migrate-data
  $ rm -rf edx-migrate-data/var/staticfiles/*

    • Approach B (Self-contained Bitnami installations):

        $ cp /opt/bitnami/apps/edx/conf/*.json edx-migrate-data
  $ cp -R /opt/bitnami/apps/edx/var edx-migrate-data
  $ rm -rf edx-migrate-data/var/staticfiles/*

  • Copy edX XQueue configuration files. Depending on your installation type, execute the following commands:

    • Approach A (Bitnami installations using system packages):

        $ cp /opt/bitnami/edx/etc/xqueue.yml edx-migrate-data

    • Approach B (Self-contained Bitnami installations):

        $ cp /opt/bitnami/apps/xqueue/conf/xqueue.yml edx-migrate-data

  • Compress the generated files:

      $ sudo tar czf edx-migrate-data.tar.gz edx-migrate-data

  • Copy edx-migrate-data.tar.gz over to the new edX instance.

Import data in the new instance

Next, log in to your machine with the latest version of Open edX installed, and navigate to the directory where you uploaded edx-migrate-data.tar.gz. Then, proceed with the following steps:

Approach A: Bitnami installations using system packages

  • Extract the compressed file:

      $ sudo tar xzf edx-migrate-data.tar.gz

  • Migrate databases:

    • MySQL:

        $ mysql -u root -p < edx-migrate-data/bitnami_edx_mysql.sql

    • MongoDB:

        $ mongorestore --authenticationDatabase admin -h localhost -u root --drop -d bitnami_edxapp edx-migrate-data/bitnami_edx_mongodb/bitnami_edxapp
  $ mongorestore --authenticationDatabase admin -h localhost -u root --drop -d bitnami_cs_comments_service edx-migrate-data/bitnami_edx_mongodb/bitnami_cs_comments_service

  • Migrate edX Platform data:

    • Replace the edX Platform data directory with the one copied from your edX Ginkgo/Hawthorn installation:

        $ sudo rm -rf /opt/bitnami/edx/var
  $ sudo cp -R edx-migrate-data/var /opt/bitnami/edx/var
  $ sudo chown -R daemon:daemon /opt/bitnami/edx/var

    • (Optional) Review edX Platform configuration for custom changes in the previous installation (e.g. themes, SMTP configuration, etc.):

      • LMS configuration file: /opt/bitnami/edx/etc/lms.yml.
      • CMS configuration file: /opt/bitnami/edx/etc/studio.yml.

    • Drop tables used by djcelery (which should already be empty in your Ginkgo data):

    • Run edX Platform migrations, which will update your Ginkgo/Hawthorn data to be valid for the current Open edX release:

        $ sudo edxapp-migrate-lms
  $ sudo edxapp-migrate-cms

    • Update assets for edX Platform:

        $ sudo edxapp-update-assets

  • Migrate edX XQueue data:

    • (Optional) Review XQueue configuration for custom changes in your previous installation: /opt/bitnami/edx/etc/xqueue.yml.

      NOTE: Since edX Hawthorn, a couple of major changes have happened: The XQueue configuration files xqueue.auth.json and xqueue.env.json have been merged into xqueue.yml, and it does not use RabbitMQ anymore.

    • Change to the root system user:

        $ sudo su

    • Run edX XQueue migrations, which will update your XQueue data to be valid for the latest version of Open edX:

        $ cd /opt/bitnami/edx/app/xqueue/xqueue
  $ source /opt/bitnami/edx/app/xqueue/xqueue_env
  $ /opt/bitnami/xqueue/bin/django-admin.py migrate --pythonpath=/opt/bitnami/app/xqueue/xqueue

  • Restart all services:

      $ sudo /opt/bitnami/ctlscript.sh restart

Your Bitnami edX stack should now be fully migrated. Confirm that the migration was successful by logging into edX and verifying the integrity of your data.

Approach B: Self-contained Bitnami installations

  • Extract the compressed file:

      $ sudo tar xzf edx-migrate-data.tar.gz

  • Migrate databases:

    • MySQL:

        $ mysql -u root -p < edx-migrate-data/bitnami_edx_mysql.sql

    • MongoDB:

        $ mongorestore --authenticationDatabase admin -h localhost -u root --drop -d bitnami_edxapp edx-migrate-data/bitnami_edx_mongodb/bitnami_edxapp
  $ mongorestore --authenticationDatabase admin -h localhost -u root --drop -d bitnami_cs_comments_service edx-migrate-data/bitnami_edx_mongodb/bitnami_cs_comments_service

  • Migrate edX Platform data:

    • Replace the edX Platform data directory with the one copied from your previous edX installation:

        $ sudo rm -rf /opt/bitnami/apps/edx/var
  $ sudo cp -R edx-migrate-data/var /opt/bitnami/apps/edx/var
  $ sudo chown -R daemon:daemon /opt/bitnami/apps/edx/var

    • (Optional) Review edX Platform configuration for custom changes in the previous installation (e.g. themes, SMTP configuration, etc.):

      • LMS configuration file: /opt/bitnami/apps/edx/conf/lms.yml.
      • CMS configuration file: /opt/bitnami/apps/edx/conf/cms.yml.

    • Run edX Platform migrations:

        $ sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-lms
  $ sudo /opt/bitnami/apps/edx/bin/edxapp-migrate-cms

    • Update assets for edX Platform:

        $ sudo /opt/bitnami/apps/edx/bin/edxapp-update-assets-lms
  $ sudo /opt/bitnami/apps/edx/bin/edxapp-update-assets-cms

  • Migrate edX XQueue data:

    • (Optional) Review XQueue configuration for custom changes in your previous installation: /opt/bitnami/apps/xqueue/conf/xqueue.yml.

      NOTE: Since edX Hawthorn, a couple of major changes have happened: The XQueue configuration files xqueue.auth.json and xqueue.env.json have been merged into xqueue.yml, and it does not use RabbitMQ anymore.

    • Run edX XQueue migrations:

        $ sudo /opt/bitnami/apps/xqueue/bin/django-admin.xqueue migrate --noinput

  • Restart Apache, edX Celery Workers and XQueue:

      $ sudo /opt/bitnami/ctlscript.sh restart apache
  $ sudo /opt/bitnami/ctlscript.sh restart edx
  $ sudo /opt/bitnami/ctlscript.sh restart xqueue

Your Bitnami edX stack should now be fully migrated. Confirm that the migration was successful by logging into edX and verifying the integrity of your data.

Last modification March 15, 2022