Rbd

From Ceph wiki

Introduction

The linux kernel RBD (rados block device) driver allows striping a linux block device over multiple distributed object store data objects. It is compatible with the kvm rbd image.

Currently the linux kernel rbd utilizes the libceph module. Both are found in the "rbd" branch of ceph-client.git. RBD is available in Linux kernels 2.6.37 and higher.

Warning: Don't use rbd kernel driver on the osd server. Perhaps it will freeze the rbd client and your osd server. You can use QEMU-RBD anyway. See also QEMU-RBD.

Checkout

git clone git://github.com/NewDreamNetwork/ceph-client.git
cd ceph-client
git checkout -b unstable origin/unstable

Compiling

Checkout the latest unstable branch and do:

make EXTRA_CFLAGS="-DCONFIG_BLK_DEV_RBD -DCONFIG_CEPH_LIB"
make modules_install EXTRA_CFLAGS="-DCONFIG_BLK_DEV_RBD -DCONFIG_CEPH_LIB"
depmod -a
modprobe rbd

You should now have a /sys/bus/rbd directory.

Setup

An rbd image can either be created using the kvm rbd tools, or by using the rbd tool. Here we create a 1GB rbd image named 'foo':

$ rbd create foo --size 1024

If you want to resize the image ,you can do this :

$ rbd resize --image foo --size 2048 (here we resize it to 2G)

Remove image:

$ rbd rm foo

You can list current rbd images with

$ rbd list
foo

Then, on the client side the ceph module needs to be loaded:

$ modprobe rbd

Here's an example for mapping the device, where 10.0.1.2 and 10.0.1.3 are the ceph monitor addresses, we use the name=admin 'mount' options, the 'rbd' pool name and the 'foo' rbd image:

$ echo "10.0.1.2,10.0.1.3 name=admin rbd foo" > /sys/bus/rbd/add

If authentication is required the command will be:

$ echo "10.0.1.2,10.0.1.3 name=admin,secret=xyz rbd foo" > /sys/bus/rbd/add

The xyz key should be replaced with the key obtained using the cauthtool on one of the monitors keyring data,not the file name:

$ cauthtool --name=admin --print-key /etc/ceph/keyring.bin > /path/to/secretfile

and in this condition, `cat /path/to/secret ` = xyz

List currently mapped rbd devices:

$ ls /sys/bus/rbd/devices

This will show you a numeric directory for each RBD device, this contains information about the device (See below for more information).

If /dev/rbd0 doesn't exist (it will usually get created for you on modern distributions), take the major number from the above (254 in this case), and excute command below:

$ mknod /dev/rbd0 b 254 0

Now the device is running and ready. Let's create a file system and mount it:

$ mkfs -t ext3 /dev/rbd0
$ mount -t ext3 /dev/rbd0 /mnt

Device information

Every RBD device has it's own directory with information, rbd0 has /sys/bus/rbd/devices/0 which has:

• client_id
• create_snap
• current_snap
• major
• name
• pool
• refresh
• rollback_snap
• size

With these files you can control the device, for example creating a snapshot.

Removing a device

Simply echo the device number into /sys/bus/rbd/remove

$ echo 0 > /sys/bus/rbd/remove

Snaphots

Snapshots are read-only and can not be created from another snapshot.

Creating a device out of a snapshot is done in the same manner as creating a regular device. Add the snap name as an additional argument as shown below:

$ echo "10.0.1.2,10.0.1.3 name=admin rbd foo sn1" > /sys/bus/rbd/add

Creating a snapshot requires that the device points to the head (not an existing snapshot):

$ echo sn2 > /sys/bus/rbd/devices/0/create_snap

The image contents can be rolled back to a previously created snapshot. This is not necessarily fast (it is O(n), where n is the number of objects backing the rbd image), and discards the current head content (snapshots are untouched).

$ echo "sn2" > /sys/bus/rbd/devices/0/rollback_snap