From Ceph wiki
The linux kernel RBD (rados block device) driver allows striping a linux block device over multiple distributed object store data objects. It is compatible with the kvm rbd image.
Currently the linux kernel rbd utilizes the libceph module. Both are found in the "rbd" branch of ceph-client.git. RBD is available in Linux kernels 2.6.37 and higher.
git clone git://github.com/NewDreamNetwork/ceph-client.git cd ceph-client git checkout -b unstable origin/unstable
Checkout the latest unstable branch and do:
make EXTRA_CFLAGS="-DCONFIG_BLK_DEV_RBD -DCONFIG_CEPH_LIB" make modules_install EXTRA_CFLAGS="-DCONFIG_BLK_DEV_RBD -DCONFIG_CEPH_LIB" depmod -a modprobe rbd
You should now have a /sys/bus/rbd directory.
An rbd image can either be created using the kvm rbd tools, or by using the rbd tool. Here we create a 1GB rbd image named 'foo':
$ rbd create foo --size 1024
If you want to resize the image ,you can do this :
$ rbd resize --image foo --size 2048 (here we resize it to 2G)
$ rbd rm foo
You can list current rbd images with
$ rbd list foo
Then, on the client side the ceph module needs to be loaded:
$ modprobe rbd
Here's an example for mapping the device, where 10.0.1.2 and 10.0.1.3 are the ceph monitor addresses, we use the name=admin 'mount' options, the 'rbd' pool name and the 'foo' rbd image:
$ echo "10.0.1.2,10.0.1.3 name=admin rbd foo" > /sys/bus/rbd/add
If authentication is required the command will be:
$ echo "10.0.1.2,10.0.1.3 name=admin,secret=xyz rbd foo" > /sys/bus/rbd/add
The xyz key should be replaced with the key obtained using the cauthtool on one of the monitors keyring data,not the file name:
$ cauthtool --name=admin --print-key /etc/ceph/keyring.bin > /path/to/secretfile
and in this condition, `cat /path/to/secret ` = xyz
List currently mapped rbd devices:
$ ls /sys/bus/rbd/devices
This will show you a numeric directory for each RBD device, this contains information about the device (See below for more information).
If /dev/rbd0 doesn't exist (it will usually get created for you on modern distributions), take the major number from the above (254 in this case), and excute command below:
$ mknod /dev/rbd0 b 254 0
Now the device is running and ready. Let's create a file system and mount it:
$ mkfs -t ext3 /dev/rbd0 $ mount -t ext3 /dev/rbd0 /mnt
Every RBD device has it's own directory with information, rbd0 has /sys/bus/rbd/devices/0 which has:
With these files you can control the device, for example creating a snapshot.
Removing a device
Simply echo the device number into /sys/bus/rbd/remove
$ echo 0 > /sys/bus/rbd/remove
Snapshots are read-only and can not be created from another snapshot.
Creating a device out of a snapshot is done in the same manner as creating a regular device. Add the snap name as an additional argument as shown below:
$ echo "10.0.1.2,10.0.1.3 name=admin rbd foo sn1" > /sys/bus/rbd/add
Creating a snapshot requires that the device points to the head (not an existing snapshot):
$ echo sn2 > /sys/bus/rbd/devices/0/create_snap
The image contents can be rolled back to a previously created snapshot. This is not necessarily fast (it is O(n), where n is the number of objects backing the rbd image), and discards the current head content (snapshots are untouched).
$ echo "sn2" > /sys/bus/rbd/devices/0/rollback_snap