CephFS Admin Tips – Create a new user and share

Hi my name is Stephen McElroy, and in this guide I will be showing how to create a new user, set permissions, set quotas, mount the share, and make them persistent on the client.

Creating the user

On the Ceph admin nodeLets create a basic user and give it capabilities to read the / and the /test_folder in CephFS.

1
2
3
4
5
6
7
8
9
10
11

$ ceph-authtool --create-keyring /etc/ceph/ceph.client.test_user.keyring --gen-key -n client.test_user
$ vi /etc/ceph/ceph.client.test_user.keyring
# Initial keyring only has key value
[client.test_user]
    key = AQAX4PBZw5tcGhAaaaaaBCSJR8qZ25uQB3yYA2gw==
    
# We will add in our capabilities here
    caps mds = "allow r path=/, allow rw path=/test_folder"
    caps mon = "allow r"
    caps osd = "allow class-read object_prefix rbd_children, allow rw pool=cephfs_data"

Once we are done adding capabilities, we will use ceph auth import to update -or- create our user entry. I personally like this way of updating the capabilities for a user for two reasons. First, it allows me to backup clients CAPS, most importantly, It allows me to not accidentally override their CAPS with ceph auth caps command.

1

$ ceph auth import -i /etc/ceph/ceph.client.test_user.keyring

Creating the CephFS share

If you don’t already have CephFS mounted somewhere to be able to create directories, lets mount the root directory now. Then create a subdirectory names test_folder.

Note – If you want to set user quotas on directory, use ceph-fuse when mounting. So far its the only way I’ve been able to get quotas to work.

1
2
3

$ mkdir /mnt/cephfs
$ ceph-fuse /mnt/cephfs
$ mkdir /mnt/cephfs/test_folder

Lets set a quota on test_folder.

1
2

$ cd /mnt/cephfs/
$ setfattr -n ceph.quota.max_bytes -v 107300000000 test_folder

Lets mount up the test folder to ensure quotas worked.

1
2
3
4

$ ceph-fuse -r /test_folder /mnt/cephfs
$ df -h
~~~
ceph-fuse                     100G     0  100G   0% /mnt/cephfs

Next, install packages for ceph-fuse

1

$ yum install ceph-fuse

Create Mount Points

Copy over your client key you made on the admin node, and ceph.conf, to “/etc/ceph/“
Then we will make two directories that will be use for mounting CephFS.
Personally I like to keep the mount directory and Ceph directory name the same.

1
2

$ mkdir /etc/cephfs_root
$ mkdir /etc/test_folder

Make this a persistent mount by adding entries in “/etc/fstab”. Change the information as needed.

1
2
3
4
5

# Ceph Fuse mount of root cephfs
id=test_user,conf=/etc/ceph/ceph.conf,client_mountpoint=/ /mnt/cephfs_root fuse.ceph noatime 0 0
# Specific Directory in cephfs
id=test_user,conf=/etc/ceph/ceph.conf,client_mountpoint=/test_folder /mnt/test_folder fuse.ceph noatime 0 0

Run mount -a and df -h to ensure everything mounted correctly.

1
2
3
4
5

$ mount -a
$ df -h
~~~
ceph-fuse                     4.2E     0  4.2E   0% /mnt/cephfs_root
ceph-fuse                     100G     0  100G   0% /mnt/test_folder

Fin

There you have it, you should now have a fully working CephFS share. I hope this helps out peeps and makes like a little easier. If this even helped out one admin, then it was well worth it. If you have any questions, or need to hire a Ceph Engineer, free to contact me at magusnebula@gmail.com!

Source: Stephen McElroy (CephFS Admin Tips – Create a new user and share)