Skip to content

AWS CLI & IAM Features

The AWS Command Line Interface (CLI) is the standard tool for managing S3-compatible object storage. Ceph S3 supports a wide range of AWS S3 and IAM (Identity and Access Management) API calls.

This document guides you through the installation of the AWS CLI and demonstrates how to leverage Ceph's IAM features to manage granular access permissions for users and applications.

1. AWS CLI Installation and Preparation

Since system packages and Python modules in newer Linux distributions often conflict with each other, we recommend installing within a virtual Python environment (venv). This prevents conflicts with the operating system.

# Install system packages for Python pip and venv (example for Debian/Ubuntu)
sudo apt update
sudo apt install python3-pip python3-venv

# Create virtual environment (e.g., in the home directory)
python3 -m venv ~/ceph.venv

# Activate virtual environment
source ~/ceph.venv/bin/activate

# Install or upgrade pip, wheel, and the required AWS/Ceph tools
pip install -U pip wheel
pip install -U awscli s3cmd boto3

[!IMPORTANT] Note: You must activate the virtual environment in every new terminal session (source ~/ceph.venv/bin/activate) before using the aws commands.

2. AWS CLI Configuration for Ceph S3

Since we are not using the AWS-native network, we need to specify our specific Ceph S3 endpoints to the CLI. We recommend using environment variables.

Replace xxx with your Access Key and Secret Key, which you generated in the Cloud Services Portal.

# Credentials
export AWS_ACCESS_KEY_ID="xxx"
export AWS_SECRET_ACCESS_KEY="xxx"

# Ceph-specific fixes for checksums
export AWS_REQUEST_CHECKSUM_CALCULATION=when_required
export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required

# Define S3 endpoint and region (example ch-zh1)
export url=https://s3.ewstorage.ch 
export region=ch-zh1 

# Or for Region2:
export url=https://s3.ch-ge1.ewstorage.ch 
export region=ch-ge1 

# Test: List your own buckets
aws --endpoint=${url} --region=${region} s3 ls

3. IAM (Identity and Access Management) in Ceph

Ceph supports core AWS IAM API functions. This enables you to assign fine-grained access permissions. Instead of granting full access (using your main keys) to every system, you can create dedicated "IAM Users" for specific applications (e.g., backup scripts, web apps) and restrict their permissions via "User Policy".

3.1 Creating a New IAM User

Create a new user (e.g., app-backup-user). This user exists logically under your main account.

aws --endpoint=${url} --region=${region} iam create-user \
    --user-name app-backup-user

3.2 Generating Access Keys for the IAM User

For the new user to authenticate, we need to create access keys for them:

aws --endpoint=${url} --region=${region} iam create-access-key \
    --user-name app-backup-user

[!NOTE] Record the displayed values for AccessKeyId and SecretAccessKey. The Secret Key will only be shown in full this one time!

3.3 Creating and Assigning a User Policy

By default, the new IAM User has no permissions. We must explicitly grant them rights. Unlike a Bucket Policy (which is attached to the bucket), a User Policy is attached directly to the user.

Let's create a file policy.json. This policy grants the user full S3 access, but only to a specific bucket named mein-backup-bucket.

cat <<EOF > policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::mein-backup-bucket",
        "arn:aws:s3:::mein-backup-bucket/*"
      ]
    }
  ]
}
EOF

Now let's assign this policy to the user:

aws --endpoint=${url} --region=${region} iam put-user-policy \
    --user-name app-backup-user \
    --policy-name BackupBucketAccess \
    --policy-document file://policy.json

3.4 Verifying IAM Policies

You can verify the assigned permissions at any time:

# List all policies for a user
aws --endpoint=${url} --region=${region} iam list-user-policies \
    --user-name app-backup-user

# View a specific policy
aws --endpoint=${url} --region=${region} iam get-user-policy \
    --user-name app-backup-user \
    --policy-name BackupBucketAccess

With these IAM features, you can perfectly implement the Principle of Least Privilege in your Ceph S3 storage.