Swift to S3 Protocol Migration Guide

• ⚠ IMPORTANT: Ensure you have admin access to the migration host and network reachability to both endpoints before starting.

STEP 1: Identify & Inventory Your Swift Containers

Gather your Swift credentials (Tenant ID, Username, Password) and list all containers before touching any data.

1.1 List All Swift Containers

export OS_AUTH_URL=https://api-object.bluvalt.com:8083/auth/v1.0
export OS_USERNAME=<TenantID>:<username>
export OS_PASSWORD=<password>
export OS_IDENTITY_API_VERSION=1

List containers

swift -A https://api-object.bluvalt.com:8083/auth/v1.0 \
  -U <TenantID>:<username> -K <password> list

• 💡 TIP: Record all container names and sizes in a tracking sheet — you will map each one to an S3 bucket in Step 2.

STEP 2: Create S3 Buckets & Generate Access Credentials

Log in to the Bluvalt Admin Dashboard and create one S3 bucket per Swift container identified in Step 1.

• ⚠ IMPORTANT: Bucket names must be globally unique, lowercase, 3–63 characters, letters/numbers/hyphens only.

2.1 Verify Bucket Access with AWS CLI

Install AWS CLI (Ubuntu)

sudo apt install awscli

Configure Bluvalt profile

aws configure --profile bluvalt
  AWS Access Key ID:     <your-access-key-id>
  AWS Secret Access Key: <your-secret-access-key>
  Default region name:   us-east-1
  Default output format: json

Verify: list buckets

aws s3 ls --endpoint-url https://api-object.bluvalt.com:8082 --profile bluvalt

Reference: How to Install Object Storage Clients

STEP 3: Install & configure rclone

rclone is the recommended migration tool — it handles multipart uploads, checksums, bandwidth throttling, and resumable transfers.

3.1 Install rclone

curl https://rclone.org/install.sh | sudo bash
rclone version   # Confirm v1.66+

3.2 Configure Swift Source Remote (~/.config/rclone/rclone.conf)

[swift-src]
type          = swift
env_auth      = false
auth_version  = 1
auth          = https://api-object.bluvalt.com:8083/auth/v1.0
user          = <TenantID>:<username>
key           = <password>
tenant        = <project-name>
tenant_domain = Default
domain        = Default
region        = RegionOne
endpoint_type = public
chunk_size    = 5Gi

3.3 Configure S3 Destination Remote (append to rclone.conf)

[s3-dest]
type                = s3
provider            = Other
env_auth            = false
access_key_id       = <your-access-key-id>
secret_access_key   = <your-secret-access-key>
endpoint            = https://api-object.bluvalt.com:8082
region              = us-east-1
acl                 = private
chunk_size          = 8Mi
upload_concurrency  = 4
force_path_style    = true

3.4 Test Both Remotes

rclone lsd swift-src:   # Lists Swift containers
rclone lsd s3-dest:     # Lists S3 buckets

STEP 4 Migrate Data — Swift to S3

4.1  Dry Run (Recommended First)
rclone copy swift-src:<container> s3-dest:<bucket> \
  --dry-run --progress --verbose

4.2 Full Copy Migration

# Safe, non-destructive — leaves Swift source untouched
rclone copy swift-src:<container> s3-dest:<bucket> \
  --progress \
  --transfers 8 \
  --checkers 16 \
  --s3-upload-concurrency 4 \
  --log-file /var/log/rclone-migrate.log \
  --log-level INFO

• 💡 TIP: Repeat Steps 4.1 and 4.2 for every container → bucket pair listed in your tracking sheet.

STEP 5: Validate Migration Integrity

Never assume success without verification. rclone check computes checksums on both sides.

5.1 Full Checksum Comparison

rclone check swift-src:<container> s3-dest:<bucket> --progress
# ✔ Expected: 0 differences found

5.2 Find Missing Objects

rclone check swift-src:<container> s3-dest:<bucket> \
  --one-way --missing-on-dst /tmp/missing_in_s3.txt
# If file is non-empty, re-run rclone copy for missing objects

5.3 Object Count Comparison

echo -n 'Swift count: '; rclone ls swift-src:<container> | wc -l
echo -n 'S3 count:    '; rclone ls s3-dest:<bucket>     | wc -l
# Counts must match before proceeding to Step 6

STEP 6: Incremental Sync & Application Cutover

After the initial bulk copy, new objects may have landed on Swift. Run an incremental sync before flipping your applications to S3.

6.1 Incremental Sync Command

• ⚠ sync DELETES objects in dest that are not in source

rclone sync swift-src:<container> s3-dest:<bucket> \
  --progress \
  --transfers 8 \
  --log-file /var/log/rclone/final-sync.log \
  --log-level INFO \
  --backup-dir s3-dest:<bucket>-backup   # optional safety net

6.2 Application Cutover — Update Environment Variables BEFORE (Swift)

export OS_AUTH_URL=https://api-object.bluvalt.com:8083/v1
export OS_USERNAME=<TenantID>:<username>
export OS_PASSWORD=<password>
export OS_PROJECT_NAME=<project>

AFTER (S3)

export AWS_ACCESS_KEY_ID=<your-access-key-id>
export AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
export AWS_DEFAULT_REGION=us-east-1
export S3_ENDPOINT_URL=https://api-object.bluvalt.com:8082

Quick Reference — rclone Commands

Common Errors & Fixes

Useful References

• How to Install Object Storage Clients
• Bluvalt Object Storage Subscription
• Bluvalt CLI Client Setup
• rclone Documentation

• Need Help? Contact our Support Team: https://azer.solutions.com.sa/
• Please have your Tenant name & ID ready.