How to Take a Scheduled Daily Volumes Snapshots

Preface

This guide describe how to take a scheduled daily volumes snapshots from all or some volumes on STC Jeddah cloud and retaining only N days. The following steps was tested on ubuntu 16.04 server that has openstack clients installed.

ssh to the server

Install Openstack Clients in Ubuntu

sudo snap install openstackclients --classic

mkdir /home/ubuntu/volumes-snapshots

cd /home/ubuntu/volumes-snapshots

To access STC Jeddah Cloud JED1

Create openrc.sh file using your favorite editor

nano openrc.sh

Copy the following text to the file and change the Username and the Project name to yours (without the quotation marks), then save and exit.


export OS_USERNAME='Enter Your Username'
echo "enter the password"
read -sr OS_PASSWORD
export OS_PASSWORD=$OS_PASSWORD
export OS_TENANT_NAME='Enter Your Project Name'
export OS_AUTH_URL=https://api-jed1-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=jed1
export OS_PROJECT_DOMAIN_NAME=jed1

Then run

 source openrc.sh

To access STC Riyadh2 Cloud RUH2

Create openrc.sh file using your favorite editor

nano openrc.sh

Copy the following text to the file and change username, password and project name to yours (without the quotation marks), then save and exit.

export OS_USERNAME='Enter Your Username'
echo "enter the password"
read -sr OS_PASSWORD
export OS_PASSWORD=$OS_PASSWORD
export OS_TENANT_NAME='Enter Your Project Name'
export OS_AUTH_URL=https://api-ruh2-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=ruh2
export OS_PROJECT_DOMAIN_NAME=ruh2

Then run

 source openrc.sh

Taking a Scheduled Daily Volumes Snapshots

List all volumes IDs on the project and save it to volumes-IDs file:

/snap/bin/openstack volume list |awk -F '|' '{print $2}'|grep -v ID|grep -v -e '^$'>/home/ubuntu/volumes-snapshots/volumes-IDs

You can exclude any volumes from this list by removing its ID from volumes-IDs file.

Create a file named snapshots-creation. sh:

nano snapshots-creation.sh

If Your Volumes in STC Jeddah Cloud JED1: Insert the following script to the file:

#!/bin/sh
PATH=$PATH
export OS_USERNAME='username'
export OS_PASSWORD='password'
export OS_TENANT_NAME='project name'
export OS_AUTH_URL=https://api-jed1-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=jed1
export OS_PROJECT_DOMAIN_NAME=jed1
for i in $(cat /home/ubuntu/volumes-snapshots/volumes-IDs); do /snap/bin/openstack volume snapshot create --force --volume $i `date +%F`-$i-snapshot |grep -m1 id|awk -F '|' '{print $3}'|grep -v -e '^$'; sleep 20s; done >>/home/ubuntu/volumes-snapshots/`date +%F`-snapshots

If Your Volumes in STC Riyadh CLOUD RUH2: Insert the following script to the file:

PATH=$PATH
export OS_USERNAME='username'
export OS_PASSWORD='password'
export OS_TENANT_NAME='project name'
export OS_AUTH_URL=https://api-ruh2-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=ruh2
export OS_PROJECT_DOMAIN_NAME=ruh2
for i in $(cat /home/ubuntu/volumes-snapshots/volumes-IDs); do /snap/bin/openstack volume snapshot create --force --volume $i `date +%F`-$i-snapshot |grep -m1 id|awk -F '|' '{print $3}'|grep -v -e '^$'; sleep 20s; done >>/home/ubuntu/volumes-snapshots/`date +%F`-snapshots

NOTE: Don’t forget to remove the quotation marks.

Now, save and exit.

Create a file named old-snapshots-cleaning.sh

Create old snapshots cleaning script that will remove any older than N days snapshots which was created by first script (snapshots-creation .sh), change username, password and project name to yours, then save it as old-snapshots-cleaning .sh file.

nano old-snapshots-cleaning.sh

If Your Volumes in STC Jeddah CLOUD JED1: Insert the following script to the file:

#!/bin/sh
PATH=$PATH
export OS_USERNAME='username'
export OS_PASSWORD='password'
export OS_TENANT_NAME='project name'
export OS_AUTH_URL=https://api-jed1-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=jed1
export OS_PROJECT_DOMAIN_NAME=jed1
for i in $(cat /home/ubuntu/volumes-snapshots/$(date -d 'now -N days' +%F)-snapshots); do /snap/bin/openstack volume snapshot delete $i; sleep 10s; done

If Your Volumes in STC Riyadh2 CLOUD RUH2: Insert the following script to the file:

PATH=$PATH
export OS_USERNAME='username'
export OS_PASSWORD='password'
export OS_TENANT_NAME='project name'
export OS_AUTH_URL=https://api-ruh2-vdc.bluvalt.com/identity/v3
export OS_IDENTITY_API_VERSION=3
export OS_USER_DOMAIN_NAME=ruh2
export OS_PROJECT_DOMAIN_NAME=ruh2
for i in $(cat /home/ubuntu/volumes-snapshots/$(date -d 'now -N days' +%F)-snapshots); do /snap/bin/openstack volume snapshot delete $i; sleep 10s; done

NOTE: Replace N in ‘now -N days’ in last command with number of days you need to retain the created snapshots.THIS SCRIPT ONLY WORKS WITH DAILY SNAPSHOTS.

Cronjob

Add a crontab job to execute the scripts at your convenient time Please don’t forget to calculate the needed quota and request to increase storage quota if needed.

chmod a+x /home/ubuntu/volumes-snapshots/snapshots-creation.sh
chmod a+x /home/ubuntu/volumes-snapshots/old-snapshots-cleaning.sh

sudo crontab -e

Now, create the cronjob with specifying the time you need the volumes snapshots to be taken at and store logs in /var/log/cron-job.log file:

30 01 * * *  /home/ubuntu/volumes-snapshots/snapshots-creation.sh  >> /var/log/cron-job.log 2>&1
30 18 * * * /home/ubuntu/volumes-snapshots/old-snapshots-cleaning.sh  >> /var/log/cron-job.log 2>&1