crushtool is a utility that lets you create, compile, and decompile CRUSH map files.
CRUSH is a pseudo-random data distribution algorithm that efficiently maps input values (typically data objects) across a heterogeneous, hierarchically structured device map. The algorithm was originally described in detail in the following paper (although it has evolved some since then):
The tool has four modes of operation.
will compile a plaintext map.txt into a binary map file.
will take the compiled map and decompile it into a plaintext source file, suitable for editing.
will create a relatively generic map with the given layer structure. See below for examples.
will perform a dry run of a CRUSH mapping for a range of input object names, see crushtool –help for more information.
The build mode will generate relatively generic hierarchical maps. The first argument simply specifies the number of devices (leaves) in the CRUSH hierarchy. Each layer describes how the layer (or raw devices) preceding it should be grouped.
Each layer consists of:
name ( uniform | list | tree | straw ) size
The first element is the name for the elements in the layer (e.g. “rack”). Each element’s name will be append a number to the provided name.
The second component is the type of CRUSH bucket.
The third component is the maximum size of the bucket. If the size is 0, a single bucket will be generated that includes everything in the preceding layer.
Suppose we have two rows with two racks each and 20 nodes per rack. Suppose each node contains 4 storage devices for Ceph OSD Daemons. This configuration allows us to deploy 320 Ceph OSD Daemons. Lets assume a 42U rack with 2U nodes, leaving an extra 2U for a rack switch.
To reflect our hierarchy of devices, nodes, racks and rows, we would execute the following:
crushtool -o crushmap --build --num_osds 320 node straw 4 rack straw 20 row straw 2
To adjust the default (generic) mapping rules, we can run:
# decompile crushtool -d crushmap -o map.txt # edit vi map.txt # recompile crushtool -c map.txt -o crushmap
crushtool is part of the Ceph distributed file system. Please refer to the Ceph documentation at http://ceph.com/docs for more information.