Planet Ceph

Aggregated news from external sources

January 27, 2017

Erasure Code on Small Clusters

Erasure code is rather designed for clusters with a sufficient size. However if you want to use it with a small amount of hosts you can also adapt the crushmap for a better matching distribution to your need.

Here a first example for distributing data with 1 host OR 2 drive fault tolerance with k=4, m=2 on 3 hosts and more.

1
2
3
4
5
6
7
8
9
10
rule erasure_ruleset {
  ruleset X
  type erasure
  min_size 6
  max_size 6
  step take default
  step choose indep 3 type host
  step choose indep 2 type osd
  step emit
}

Testing it on sample crushmap :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# crushtool --test -i crushmap.bin --rule 1 --show-mappings --x 1 --num-rep 6
CRUSH rule 1 x 1 [0,1,8,7,5,3]

# crushtool --test -i crushmap.bin --tree
ID    WEIGHT  TYPE NAME
-5      4.00000 root default
-1      1.00000         host host-01
0       1.00000                 osd.0   <-- DATA
1       1.00000                 osd.1   <-- DATA
2       1.00000                 osd.2
-2      1.00000         host host-02
3       1.00000                 osd.3   <-- PARITY
4       1.00000                 osd.4
5       1.00000                 osd.5   <-- PARITY
-3      1.00000         host host-03
6       1.00000                 osd.6
7       1.00000                 osd.7   <-- DATA
8       1.00000                 osd.8   <-- DATA
-4      1.00000         host host-04
9       1.00000                 osd.9
10      1.00000                 osd.10
11      1.00000                 osd.11

Here is an other example for distributing data with ( 1 host AND 1 drive ) OR ( 3 drives ) fault tolerance with k=5, m=3 on 4 hosts and more.

1
2
3
4
5
6
7
8
9
10
rule erasure_ruleset {
  ruleset X
  type erasure
  min_size 8
  max_size 8
  step take default
  step choose indep 4 type host
  step choose indep 2 type osd
  step emit
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# crushtool --test -i crushmap.bin --rule 2 --show-mappings --x 1 --num-rep 8
CRUSH rule 2 x 1 [0,1,8,7,9,11,5,3]

# crushtool --test -i crushmap.bin --tree
ID      WEIGHT  TYPE NAME
-5      4.00000 root default
-1      1.00000         host host-01
0       1.00000                 osd.0   <-- DATA
1       1.00000                 osd.1   <-- DATA
2       1.00000                 osd.2
-2      1.00000         host host-02
3       1.00000                 osd.3   <-- PARITY
4       1.00000                 osd.4
5       1.00000                 osd.5   <-- PARITY
-3      1.00000         host host-03
6       1.00000                 osd.6
7       1.00000                 osd.7   <-- DATA
8       1.00000                 osd.8   <-- DATA
-4      1.00000         host host-04
9       1.00000                 osd.9   <-- DATA
10      1.00000                 osd.10
11      1.00000                 osd.11  <-- PARITY

And a last example for distributing data with ( 1 host AND 2 drives ) OR ( 4 drives ) fault tolerance with k=8, m=4 on 4 hosts and more.

1
2
3
4
5
6
7
8
9
10
rule erasure_ruleset {
  ruleset X
  type erasure
  min_size 12
  max_size 12
  step take default
  step choose indep 4 type host
  step choose indep 3 type osd
  step emit
}

An other possibility is to use LRC Erasure code plugin :

http://docs.ceph.com/docs/master/rados/operations/erasure-code-lrc/

Source: Laurent Barbe (Erasure Code on Small Clusters)

Careers