How to Install Ceph Squid v19 RGW S3 on Bare Metal Ubuntu 24.04

In this guide, we’ll install Ceph Squid v19 on three Ubuntu 24.04 bare-metal nodes, using cephadm. We’ll deploy:

• The Ceph Dashboard (Web UI),
• A fully functional S3-compatible object storage using Ceph RGW,
• With SSL disabled on both services because HAProxy will be used for SSL termination in a future step.

What is Ceph RGW S3?

Ceph RGW (RADOS Gateway) is the component that exposes Ceph’s distributed storage via S3 and Swift-compatible RESTful APIs.

With RGW deployed, your Ceph cluster acts like an Amazon S3 clone, meaning you can use:

• s3cmd, aws-cli, rclone, minio-client, and other S3-compatible tools
• For uploading, downloading, and managing buckets and objects
• On-premises, with full control and scalability

This makes Ceph a perfect choice for private cloud, enterprise backups, dev/test object stores, or geo-replicated deployments.

Prerequisites

Before you begin, make sure:

• All nodes (ceph-1, ceph-2, ceph-3) are running Ubuntu 24.04.
• Each node has:
  • A hostname configured and resolvable via /etc/hosts or DNS.
  • At least one unused, unformatted disk (e.g., /dev/sdb) that Ceph can use for storage.

You can verify unused disks with:

lsblk

Example output:

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda      8:0    0   40G  0 disk
├─sda1   8:1    0   39G  0 part /
└─sda2   8:2    0    1G  0 part [SWAP]
sdb      8:16   0  20G  0 disk  # <-- Unused disk for Ceph OSD

1. Install cephadm on All Nodes

Run the following on all nodes:

sudo apt update
sudo apt install -y cephadm

This also installs Docker and other dependencies.

2. Enable Root SSH Login on ceph-2 and ceph-3

To allow ceph-1 to orchestrate other nodes:

Edit SSH config:

sudo vi /etc/ssh/sshd_config

Change:

PermitRootLogin prohibit-password

To:

PermitRootLogin yes

Then:

sudo systemctl restart ssh
sudo passwd root

3. Bootstrap Cluster (No SSL for Dashboard)

On ceph-1:

sudo cephadm bootstrap \
  --mon-ip 192.168.0.82 \
  --initial-dashboard-user admin \
  --initial-dashboard-password admin123

Then disable SSL:

sudo cephadm shell -- ceph config set mgr mgr/dashboard/ssl false
sudo cephadm shell -- ceph mgr module disable dashboard
sudo cephadm shell -- ceph mgr module enable dashboard

Access the dashboard at:

http://ceph-1.maksonlee.com:8080

4. Set Up SSH and Add Hosts

From ceph-1, switch to root:

sudo su -
ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-2
ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-3
exit

Then:

sudo cephadm shell -- ceph orch host add ceph-2 192.168.0.83
sudo cephadm shell -- ceph orch host add ceph-3 192.168.0.84

5. Add OSDs

sudo cephadm shell -- ceph orch daemon add osd ceph-1:/dev/sdb
sudo cephadm shell -- ceph orch daemon add osd ceph-2:/dev/sdb
sudo cephadm shell -- ceph orch daemon add osd ceph-3:/dev/sdb

6. Deploy RGW on All Nodes (Port 8081)

Avoiding conflict with Dashboard (8080):

sudo cephadm shell -- ceph orch apply rgw default --placement="ceph-1,ceph-2,ceph-3" --port=8081

7. Create an S3 User

sudo cephadm shell -- radosgw-admin user create \
  --uid="myuser" \
  --display-name="My User"

Note the access_key and secret_key.

8. Test S3 with s3cmd

Install s3cmd on a client:

sudo apt install -y s3cmd
s3cmd --configure

Use:

• S3 Endpoint: http://ceph-1.maksonlee.com:8081
• Use HTTPS: No
• Access Key / Secret Key: from previous step

Create and test:

echo "hello" > hello.txt
s3cmd mb s3://testbucket
s3cmd put hello.txt s3://testbucket/
s3cmd ls
s3cmd get s3://testbucket/hello.txt

Summary

• Ceph Squid v19 installed across 3 bare-metal Ubuntu nodes
• Dashboard running at http://ceph-1.maksonlee.com:8080
• S3-compatible object storage deployed using RGW on port 8081
• No SSL enabled, because HAProxy will be placed in front later
• Ready for integration with any S3 tools or backup systems