nativeInstalleredx

Upgrade Open edX

Before running the commands shown on this page, you should load the Bitnami stack environment by executing the installdir/use_APPNAME script (Linux and Mac OS X) or by clicking the shortcut in the Start Menu under “Start -> Bitnami APPNAME Stack -> Application console” (Windows). Learn more.

NOTE: Before proceeding to upgrade the Bitnami edX stack, ensure that you have performed a complete backup.


Upgrade edX Ginkgo to Hawthorn.2

The recommended approach to upgrade edX Ginkgo to Hawthorn is to migrate the data to a fresh installation of the Bitnami edX Hawthorn stack, and copy data and settings to it.

In order to proceed, log in to your edX Ginkgo machine and follow the steps below:

  • Create a directory where to store all the data:

    $ mkdir edx-upgrade-data
    
  • Dump databases required by edX:

    • MySQL:

      $ installdir/mysql/bin/mysqldump -u root -p --databases bitnami_edxapp bitnami_edxapp_csmh bitnami_xqueue --add-drop-database > edx-upgrade-data/bitnami_edx_mysql.sql
      
    • MongoDB:

      $ installdir/mongodb/bin/mongodump --authenticationDatabase admin -u root -p -d bitnami_edxapp --out edx-upgrade-data/bitnami_edx_mongodb
      $ installdir/mongodb/bin/mongodump --authenticationDatabase admin -u root -p -d bitnami_cs_comments_service --out edx-upgrade-data/bitnami_edx_mongodb
      
  • Copy edX Platform configuration and data files (which contain uploads, courses, etc.):

    $ cp installdir/apps/edx/conf/*.json edx-upgrade-data
    $ cp -R installdir/apps/edx/var edx-upgrade-data
    
  • Copy edX XQueue configuration files:

    $ cp installdir/apps/edx/conf/*.json edx-upgrade-data
    
  • Copy edX Forum configuration files:

    $ cp installdir/apps/forum/conf/httpd-forum.conf edx-upgrade-data
    
  • Compress the generated files:

    $ tar czf edx-upgrade-data.tar.gz edx-upgrade-data
    
  • Copy edx-upgrade-data.tar.gz over to the new edX Hawthorn machine.

Next, log in to your edX Hawthorn machine, and navigate to the directory where you copied edx-upgrade-data.tar.gz. Then, proceed with the following steps:

  • Extract the compressed file:

    $ tar xzf edx-upgrade-data.tar.gz
    
  • Migrate databases:

    • MySQL:

      $ installdir/mysql/bin/mysql -u root -p < edx-upgrade-data/bitnami_edx_mysql.sql
      
    • MongoDB:

      $ installdir/mongodb/bin/mongorestore -h localhost --authenticationDatabase admin -u root --drop -d bitnami_edxapp edx-upgrade-data/bitnami_edx_mongodb/bitnami_edxapp
      $ installdir/mongodb/bin/mongorestore -h localhost --authenticationDatabase admin -u root --drop -d bitnami_cs_comments_service edx-upgrade-data/bitnami_edx_mongodb/bitnami_cs_comments_service
      
  • Migrate edX Platform data:

    • Replace the edX Platform data directory with the one copied from your edX Ginkgo installation (note that you need to clear the assets):

      $ sudo rm -rf installdir/apps/edx/var
      $ sudo cp -R edx-upgrade-data/var installdir/apps/edx/var
      $ sudo chown -R daemon:daemon installdir/apps/edx/var
      $ sudo rm -rf installdir/apps/edx/var/staticfiles/*
      
    • (Optional) Review edX Platform configuration for custom changes in the previous installation (e.g. themes, SMTP configuration, etc.):

      • LMS configuration files: installdir/apps/edx/conf/lms.auth.json and installdir/apps/edx/conf/lms.env.json.
      • CMS configuration files: installdir/apps/edx/conf/cms.auth.json and installdir/apps/edx/conf/cms.env.json.
    • In order to upgrade from Ginkgo to Hawthorn, you need to drop tables used by djcelery (which should already be empty in your Ginkgo data):

      $ sudo installdir/apps/edx/bin/python.edxapp manage.py lms drop_djcelery_tables --settings=bitnami
      
    • Run edX Platform migrations, which will update your Ginkgo data to be valid for Hawthorn:

      $ sudo installdir/apps/edx/bin/edxapp-migrate-lms
      $ sudo installdir/apps/edx/bin/edxapp-migrate-cms
      
    • Update assets for edX Platform:

      $ sudo installdir/apps/edx/bin/edxapp-update-assets-lms
      $ sudo installdir/apps/edx/bin/edxapp-update-assets-cms
      
  • Migrate edX XQueue data:

    • (Optional) Check the edX XQueue configuration file for any changes for customizing your application: installdir/apps/xqueue/conf/xqueue.yml.

      NOTE: Since edX Hawthorn, a couple of major changes have happened: The XQueue configuration files xqueue.auth.json and xqueue.env.json have been merged into xqueue.yml, and it does not use RabbitMQ anymore.

    • Run edX XQueue migrations, which will update your Ginkgo data to be valid for Hawthorn:

      $ sudo installdir/apps/xqueue/bin/django-admin.xqueue migrate --noinput
      
  • Restart Apache, edX Celery Workers and XQueue:

    $ sudo installdir/ctlscript.sh restart apache
    $ sudo installdir/ctlscript.sh restart edx
    $ sudo installdir/ctlscript.sh restart xqueue
    

Upgrade edX Ficus.4 to Gingko.2

IMPORTANT: In order to upgrade any edX Ficus version to edX Ginkgo, you must first upgrade to edX Ficus.4

To upgrade edX Ficus.4 to Gingko.2, follow these steps:

  • Load the Bitnami stack environment, if it is not already loaded:

    $ sudo installdir/use_edx
    $ . installdir/scripts/setenv.sh
    
  • Stop all edX services:

    $ sudo installdir/ctlscript.sh stop
    
  • Install the GCC compiler.

    • Debian/Ubuntu:

      $ sudo apt-get update
      $ sudo apt-get install gcc g++
      
    • RHEL/CentOS:

      $ sudo yum group install "Development Tools"
      
  • Navigate to the application directory at installdir/apps/edx/edx-platform and save the current state of the Git repository:

    $ cd installdir/apps/edx/edx-platform
    $ git diff -- . ':!requirements' > ~/bitnami_edx-platform_changes.txt
    
  • Check out the Gingko.2 branch and apply the Bitnami changes to the Git repository:

    $ git fetch
    $ git checkout -f open-release/ginkgo.2
    $ git apply ~/bitnami_edx-platform_changes.txt
    
  • Install the necessary modules:

    $ npm install
    
  • Install edX’s Python dependencies in this order:

    $ . ../venvs/edxapp/bin/activate
    $ pip install -r --exists-action=w requirements/edx/setuptools.txt
    $ pip install -r --exists-action=w requirements/edx/pre.txt
    $ pip install -r --exists-action=w requirements/edx/github.txt
    $ pip install -r --exists-action=w requirements/edx/local.txt
    $ pip install -r --exists-action=w requirements/edx/base.txt
    $ pip install -r --exists-action=w requirements/edx/paver.txt
    $ pip install -r --exists-action=w requirements/edx/post.txt
    $ deactivate
    
  • Install edX Sandbox’s Python dependencies and fix ownerships:

    $ . ../venvs/edxapp-sandbox/bin/activate
    $ sudo pip install -r requirements/edx-sandbox/setuptools.txt
    $ sudo pip install -r requirements/edx-sandbox/base.txt --ignore-installed
    $ sudo pip install -r requirements/edx-sandbox/local.txt
    $ sudo pip install -r requirements/edx-sandbox/post.txt
    $ deactivate
    $ sudo chown -R sandbox:sandbox ../venvs/edxapp-sandbox/
    
  • Start databases:

    $ sudo installdir/ctlscript.sh start mysql
    $ sudo installdir/ctlscript.sh start mongodb
    $ sudo installdir/ctlscript.sh start memcached
    $ sudo installdir/ctlscript.sh start rabbitmq
    $ sudo installdir/ctlscript.sh start elasticsearch
    
  • Update edX assets. Note that this may take a significant amount of time for each step:

    $ ../bin/edxapp-update-assets-cms
    $ ../bin/edxapp-update-assets-lms
    
  • Drop the djcelery database tables:

    $ sudo ../bin/python.edxapp manage.py lms drop_djcelery_tables --settings=aws
    
  • Run edX migrations (this may also take some time):

    $ sudo ../bin/edxapp-migrate-cms
    $ sudo ../bin/edxapp-migrate-lms
    
  • Fix log file ownership:

    $ sudo chown -R daemon:daemon installdir/apps/edx/var/log
    
  • Edit the installdir/apps/edx/scripts/ctl-edx-celery-workers.sh script and replace this:

    start_worker() {
        ...
    }
    

    with this:

    start_worker() {
        EDX_HOME="installdir/apps/edx"
        cd $EDX_HOME/edx-platform
        APP=$(echo $1 | cut -d. -f2)
        ADDITIONAL_PARAMS=""
        if [ "$APP" = "cms" ]; then
            ADDITIONAL_PARAMS="-I contentstore.tasks"
        fi
        if [ `id|sed -e s/uid=//g -e s/\(.*//g` -eq 0 ]; then
            su -s /bin/sh daemon -c "$EDX_HOME/bin/python.edxapp ./manage.py $APP --settings=aws celery worker $ADDITIONAL_PARAMS --loglevel=info --queues=$1 --hostname=$1.%h --concurrency=$2 > $EDX_HOME/var/log/celery/$1.log 2>&1 &"
        else
            $EDX_HOME/bin/python.edxapp ./manage.py $APP --settings=aws celery worker $ADDITIONAL_PARAMS --loglevel=info --queues=$1 --hostname=$1.%h --concurrency=$2 > $EDX_HOME/var/log/celery/$1.log 2>&1 &
        fi
        cd - > /dev/null
    }
    
  • Edit the lms.env.json file and edit the CELERY_QUEUES section so that it looks like this:

    "CELERY_QUEUES": [
        "edx.lms.core.low",
        "edx.lms.core.default",
        "edx.lms.core.high",
        "edx.lms.core.high_mem"
    ],
    ...
    
  • Upgrade xqueue:

    $ cd installdir/apps/xqueue/xqueue
    $ git diff -- . ':!*requirements.txt' ':!xqueue/test_settings.py' > ~/bitnami_xqueue_changes.txt
    $ git fetch
    $ git checkout -f open-release/ginkgo.2
    $ git apply ~/bitnami_xqueue_changes.txt
    
  • Migrate and update users:

    $ sudo ../bin/django-admin.xqueue migrate --noinput
    $ sudo ../bin/django-admin.xqueue update_users
    
  • Upgrade the comments service:

    $ cd installdir/apps/forum/cs_comments_service
    $ git fetch
    $ git checkout -f open-release/ginkgo.2
    $ sed -i 's/^ruby ".*"$//' Gemfile
    $ bundle install
    
  • Start the remaining services:

    $ sudo installdir/ctlscript.sh start apache
    $ sudo installdir/ctlscript.sh start edx
    $ sudo installdir/ctlscript.sh start xqueue
    

Your Bitnami edX stack should now be upgraded to Gingko.2. Confirm that the upgrade was successful by logging into edX and verifying the integrity of your data.

Upgrade edX Ficus to Ficus.4

To upgrade edX Ficus to Ficus.4, follow these steps:

  • Load the Bitnami stack environment, if it is not already loaded:

    $ sudo installdir/use_edx
    $ . installdir/scripts/setenv.sh
    
  • Stop all edX services:

    $ sudo installdir/ctlscript.sh stop
    
  • Install the GCC compiler.

    • Debian/Ubuntu:

      $ sudo apt-get update
      $ sudo apt-get install gcc g++
      
    • RHEL/CentOS:

      $ sudo yum group install "Development Tools"
      
  • Navigate to the application directory at installdir/apps/edx/edx-platform and save the current state of the Git repository:

    $ cd installdir/apps/edx/edx-platform
    $ git diff -- . ':!requirements' > ~/bitnami_edx-platform_changes.txt
    
  • Check out the Ficus.4 branch and apply the Bitnami changes to the Git repository:

    $ git fetch
    $ git checkout -f open-release/ficus.4
    $ git apply ~/bitnami_edx-platform_changes.txt
    
  • Install the necessary modules:

    $ npm install
    
  • Install edX’s Python dependencies in this order:

    $ . ../venvs/edxapp/bin/activate
    $ pip install -r --exists-action=w requirements/edx/setuptools.txt
    $ pip install -r --exists-action=w requirements/edx/pre.txt
    $ pip install -r --exists-action=w requirements/edx/github.txt
    $ pip install -r --exists-action=w requirements/edx/local.txt
    $ pip install -r --exists-action=w requirements/edx/base.txt
    $ pip install -r --exists-action=w requirements/edx/paver.txt
    $ pip install -r --exists-action=w requirements/edx/post.txt
    $ deactivate
    
  • Install edX Sandbox’s Python dependencies and fix ownerships:

    $ . ../venvs/edxapp-sandbox/bin/activate
    $ sudo pip install -r requirements/edx-sandbox/setuptools.txt
    $ sudo pip install -r requirements/edx-sandbox/base.txt --ignore-installed
    $ sudo pip install -r requirements/edx-sandbox/local.txt
    $ sudo pip install -r requirements/edx-sandbox/post.txt
    $ deactivate
    $ sudo chown -R sandbox:sandbox ../venvs/edxapp-sandbox/
    
  • Start databases:

    $ sudo installdir/ctlscript.sh start mysql
    $ sudo installdir/ctlscript.sh start mongodb
    $ sudo installdir/ctlscript.sh start memcached
    $ sudo installdir/ctlscript.sh start rabbitmq
    $ sudo installdir/ctlscript.sh start elasticsearch
    
  • Update edX assets. Note that this may take a significant amount of time for each step:

    $ ../bin/edxapp-update-assets-cms
    $ ../bin/edxapp-update-assets-lms
    
  • Run edX migrations (this may also take some time):

    $ sudo ../bin/edxapp-migrate-cms
    $ sudo ../bin/edxapp-migrate-lms
    
  • Fix log file ownership:

    $ sudo chown -R daemon:daemon installdir/apps/edx/var/log
    
  • Edit the installdir/apps/edx/scripts/ctl-edx-celery-workers.sh script and replace this:

    start_worker() {
        ...
    }
    

    with this:

    start_worker() {
        EDX_HOME="installdir/apps/edx"
        cd $EDX_HOME/edx-platform
        APP=$(echo $1 | cut -d. -f2)
        ADDITIONAL_PARAMS=""
        if [ "$APP" = "cms" ]; then
            ADDITIONAL_PARAMS="-I contentstore.tasks"
        fi
        if [ `id|sed -e s/uid=//g -e s/\(.*//g` -eq 0 ]; then
            su -s /bin/sh daemon -c "$EDX_HOME/bin/python.edxapp ./manage.py $APP --settings=aws celery worker $ADDITIONAL_PARAMS --loglevel=info --queues=$1 --hostname=$1.%h --concurrency=$2 > $EDX_HOME/var/log/celery/$1.log 2>&1 &"
        else
            $EDX_HOME/bin/python.edxapp ./manage.py $APP --settings=aws celery worker $ADDITIONAL_PARAMS --loglevel=info --queues=$1 --hostname=$1.%h --concurrency=$2 > $EDX_HOME/var/log/celery/$1.log 2>&1 &
        fi
        cd - > /dev/null
    }
    
  • Edit the lms.env.json file and edit the CELERY_QUEUES section so that it looks like this:

    "CELERY_QUEUES": [
        "edx.lms.core.low",
        "edx.lms.core.default",
        "edx.lms.core.high",
        "edx.lms.core.high_mem"
    ],
    
  • Start remaining services:

    $ sudo installdir/ctlscript.sh start apache
    $ sudo installdir/ctlscript.sh start edx
    

Your Bitnami edX stack should now be upgraded to Ficus.4. Confirm that the upgrade was successful by logging into edX and verifying the integrity of your data.