Ceph + OpenStack :: Part-3

syndicated

Testing OpenStack Cinder + RBD

  • Creating a cinder volume provided by ceph backend

[root@rdo /]#
[root@rdo /]# cinder create --display-name cinder-ceph-vol1 --display-description "first cinder volume on ceph backend" 10
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2013-11-27T19:35:39.481075 |
| display_description | first cinder volume on ceph backend |
| display_name | cinder-ceph-vol1 |
| id | 10cc0855-652a-4a9b-baa1-80bc86dc12ac |
| metadata | {} |
| size | 10 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| volume_type | None |
+---------------------+--------------------------------------+
[root@rdo /]#
[root@rdo /]#

[root@rdo /(keystone_admin)]# cinder list
+--------------------------------------+-----------+------------------+------+--------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+------------------+------+--------------+----------+-------------+
| 10cc0855-652a-4a9b-baa1-80bc86dc12ac | available | cinder-ceph-vol1 | 5 | ceph-storage | false | |
| 9671edaa-62c8-4f98-a36c-d6e59612141b | available | boot_from_volume | 20 | None | false | |
+--------------------------------------+-----------+------------------+------+--------------+----------+-------------+
[root@rdo /(keystone_admin)]#

[root@rdo /]#

[root@rdo /]# rados lspools
data
metadata
rbd
ceph-images
ceph-volumes
[root@rdo /]#
[root@rdo /]#
[root@rdo /]# rbd -p ceph-volumes ls
volume-10cc0855-652a-4a9b-baa1-80bc86dc12ac
[root@rdo /]#

  • Attaching cinder volume to Instance

[root@rdo /(keystone_admin)]# nova list
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
| 0043a8be-60d1-43ed-ba43-1ccd0bba7559 | instance2 | SHUTOFF | None | Shutdown | public=172.24.4.228 |
| 9d3c327f-1893-40ff-8a82-16fad9ce6d91 | small-ubuntu | ACTIVE | None | Running | public=172.24.4.230 |
| 10d1c49f-9fbc-455f-b72d-f731338b2dd5 | small-ubuntu-pwd | ACTIVE | powering-off | Shutdown | public=172.24.4.231 |
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
[root@rdo /(keystone_admin)]#

[root@rdo /(keystone_admin)]# nova show 9d3c327f-1893-40ff-8a82-16fad9ce6d91
+--------------------------------------+--------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+--------------------------------------------------------------------+
| status | ACTIVE |
| updated | 2013-12-03T15:58:31Z |
| OS-EXT-STS:task_state | None |
| OS-EXT-SRV-ATTR:host | rdo |
| key_name | RDO-admin |
| image | Ubuntu 12.04 cloudimg amd64 (f61edc8d-c9a1-4ff4-b4fc-c8128bd1a10b) |
| hostId | 4a74aa79a23a084f73f49a4fedba7447c132ab45c4701ed7fbbb2286 |
| OS-EXT-STS:vm_state | active |
| OS-EXT-SRV-ATTR:instance_name | instance-00000018 |
| public network | 172.24.4.230 |
| OS-SRV-USG:launched_at | 2013-12-03T08:55:46.000000 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | rdo |
| flavor | m1.small (2) |
| id | 9d3c327f-1893-40ff-8a82-16fad9ce6d91 |
| security_groups | [{u'name': u'default'}] |
| OS-SRV-USG:terminated_at | None |
| user_id | 99f8019ba2694d78a680a5de46aa1afd |
| name | small-ubuntu |
| created | 2013-12-03T08:55:39Z |
| tenant_id | 0dafe42cfde242ddbb67b681f59bdb00 |
| OS-DCF:diskConfig | MANUAL |
| metadata | {} |
| os-extended-volumes:volumes_attached | [] |
| accessIPv4 | |
| accessIPv6 | |
| progress | 0 |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-AZ:availability_zone | nova |
| config_drive | |
+--------------------------------------+--------------------------------------------------------------------+
[root@rdo /(keystone_admin)]#

[root@rdo /(keystone_admin)]# virsh list
Id Name State
---------------------------------------------------- 2 instance-00000018 running

[root@rdo /(keystone_admin)]# cat disk.xml

\[root@rdo /(keystone\_admin)\]#

  • Things you should know about this file

    • source name=<ceph_pool_name/volume_name> ## ceph pool that we have created above and its cinder volume
    • host name=<Your_Monitor_nodes>
    • auth username=<user_you_created_in_Ceph_having_rights_to_pools_that_will_be_used_with_OS> ## we have created 2 users client.volumes and client.images for ceph that will have access to pools for openstack
    • secret uuid=<secret_generated_by_virsh_secrec_define_command> ## refer above that we have generated.

  • Attaching disk device to instance

[root@rdo /(keystone_admin)]# virsh attach-device instance-00000018 disk.xml
Device attached successfully
[root@rdo /(keystone_admin)]#

  • Now the ceph volume is attached to your openstack instance , you can use this as a regular block disk.

Making integration more seamless

  • To allow openstack create and attach ceph volumes using nova / cinder CLI as well as horizon dashboard , we need to add in /etc/nova/nova.conf the following values

rbd_user=volumes
rbd_secret_uuid=801a42ec-aec1-3ea8-d869-823c2de56b83

  • After updating nova.conf , try creating volume from nova cli and attach to instance

[root@rdo nova(keystone_admin)]#
[root@rdo nova(keystone_admin)]# nova volume-create --display_name=nova-vol_1 2
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| status | creating |
| display_name | nova-vol_1 |
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2013-12-04T14:13:07.265831 |
| display_description | None |
| volume_type | None |
| snapshot_id | None |
| source_volid | None |
| size | 2 |
| id | 0e2bfced-be6a-44ec-a3ca-22c771c66cdc |
| metadata | {} |
+---------------------+--------------------------------------+
[root@rdo nova(keystone_admin)]#

[root@rdo nova(keystone_admin)]# nova volume-list
+--------------------------------------+-----------+------------------+------+--------------+-------------+
| ID | Status | Display Name | Size | Volume Type | Attached to |
+--------------------------------------+-----------+------------------+------+--------------+-------------+
| 0e2bfced-be6a-44ec-a3ca-22c771c66cdc | available | nova-vol_1 | 2 | None | |
| 9671edaa-62c8-4f98-a36c-d6e59612141b | available | boot_from_volume | 20 | None | |
| 10cc0855-652a-4a9b-baa1-80bc86dc12ac | available | ceph-vol1 | 5 | ceph-storage | |
+--------------------------------------+-----------+------------------+------+--------------+-------------+
[root@rdo nova(keystone_admin)]#

[root@rdo nova(keystone_admin)]# nova list
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
| 0043a8be-60d1-43ed-ba43-1ccd0bba7559 | instance2 | SHUTOFF | None | Shutdown | public=172.24.4.228 |
| 9d3c327f-1893-40ff-8a82-16fad9ce6d91 | small-ubuntu | ACTIVE | None | Running | public=172.24.4.230 |
| 10d1c49f-9fbc-455f-b72d-f731338b2dd5 | small-ubuntu-pwd | ACTIVE | powering-off | Shutdown | public=172.24.4.231 |
+--------------------------------------+------------------+---------+--------------+-------------+---------------------+
[root@rdo nova(keystone_admin)]#

[root@rdo nova(keystone_admin)]# nova volume-attach 9d3c327f-1893-40ff-8a82-16fad9ce6d91 0e2bfced-be6a-44ec-a3ca-22c771c66cdc /dev/vdi
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdi |
| serverId | 9d3c327f-1893-40ff-8a82-16fad9ce6d91 |
| id | 0e2bfced-be6a-44ec-a3ca-22c771c66cdc |
| volumeId | 0e2bfced-be6a-44ec-a3ca-22c771c66cdc |
+----------+--------------------------------------+
[root@rdo nova(keystone_admin)]#
[root@rdo nova(keystone_admin)]#
[root@rdo nova(keystone_admin)]# nova volume-list
+--------------------------------------+-----------+------------------+------+--------------+--------------------------------------+
| ID | Status | Display Name | Size | Volume Type | Attached to |
+--------------------------------------+-----------+------------------+------+--------------+--------------------------------------+
| 0e2bfced-be6a-44ec-a3ca-22c771c66cdc | in-use | nova-vol_1 | 2 | None | 9d3c327f-1893-40ff-8a82-16fad9ce6d91 |
| 9671edaa-62c8-4f98-a36c-d6e59612141b | available | boot_from_volume | 20 | None | |
| 10cc0855-652a-4a9b-baa1-80bc86dc12ac | available | ceph-vol1 | 5 | ceph-storage | |
+--------------------------------------+-----------+------------------+------+--------------+--------------------------------------+
[root@rdo nova(keystone_admin)]#

Please Follow Ceph + OpenStack :: Part-4 for next step in installation