The Ceph Blog

Ceph blog stories provide high-level spotlights on our customers all over the world

September 15, 2017

New in Luminous: pool tags

Congratulations on upgrading your cluster to the Luminous release of Ceph. You have diligently followed all the steps outlined in the upgrade notes to the point where you are ready to mark the upgrade complete by executing the following:

$ ceph osd require-osd-release luminous

Once your cluster has been flagged as fully upgraded to the Luminous release, a new feature will be enabled: application pool tags.

New Health Warning

Your first encounter with the new feature might possibly be in the form of a new health warning indicating that one or more of your in-use pools is missing an application tag:

$ ceph health
 HEALTH_WARN application not enabled on 3 pool(s)
 $ ceph health detail
 HEALTH_WARN application not enabled on 3 pool(s)
 POOL_APP_NOT_ENABLED application not enabled on 3 pool(s)
     application not enabled on pool 'block-devices'
     application not enabled on pool 'custom-application'
     application not enabled on pool 'object-gateway'
     use 'ceph osd pool application enable <pool-name> <app-name>', where <app-name> is 'cephfs', 'rbd', 'rgw', or freeform for custom applications.

As indicated by the detailed health message, these warnings can be disabled by running “ceph osd pool application enable <pool-name> <app-name>” for each pool listed. The <app-name> parameter is free-form, although we recommend that you use “rbd” for RBD-related pools and “rgw” for RGW-related pools.  There should not be a need to ever manually tag a CephFS pool.

For example, the above health warnings can be cleared with

$ ceph osd pool application enable block-devices rbd
$ ceph osd pool application enable custom-application mything
$ ceph osd pool application enable object-gateway rgw

Automatic Tagging

The final Luminous upgrade step via “ceph osd require-osd-release luminous” makes an attempt to automatically tag any existing pools with the appropriate application using the following heuristics:

  • CephFS-related pools are automatically tagged with “cephfs”
  • Pool names containing “rbd” or pools named “images”, “volumes”, “backups”, or “vms” are automatically tagged with “rbd”
  • Pool names containing “.rgw”, “.log”, “.intent-log”, “.usage”, or “.users” are automatically tagged with “rgw”
  • Pools that match more than one heuristic are not automatically tagged during the upgrade.

Once the upgrade is complete, adding new pools to CephFS will automatically tag those pools as “cephfs”-owned and RGW will automatically tag any pools it creates as “rgw”-owned.

New clusters and new pools

If you create a fresh Ceph cluster with Luminous, you’ll notice another change: there no default pools.  Previously versions pre-created an “rbd” pool for you but Luminous does not presume to know what you will use the cluster for (or how many PGs are appropriate for your use-case).

If you will be using RBD, you’ll need to create the pool yourself.  You may want to refer to the discussion of placement groups to decide what value for <pg_num> to use.

$ ceph osd pool create <pool-name> <pg_num>

Once the pool is created, there is a new rbd command to simplify the tagging of new RBD pools:

$ rbd pool init <pool-name>

How are Application Pool Tags Used?

Disregarding the potentially scary health warning, the absence or presence of application pool tags will not affect the behavior of your Luminous cluster. Yet.  Pool tags were introduced into the system as a simple foundation for future features. Some of these future changes include:

  1. Simplifying the behavior of higher-level management tools by allowing them to easily determine the use-case of a pool. For example, the new Ceph Dashboard (blog coming soon) currently uses a set of heuristics to guess if a pool is used for RBD workloads; pool tags avoid that awkward and error-prone process.
  2. Prevent an application from using a pool that is not properly tagged for its use. For example, the rbd CLI could warn about or prevent the creation of images in an RGW-tagged pool.
Jason Dillaman