Planet Ceph

Aggregated news from external sources

  • September 5, 2014
    OpenStack at the CephDays Paris

    Save the date (September 18, 2014) and join us at the new edition of the Ceph Days in Paris.
    I will be talking about the new amazing stuff that happened during this (non-finished yet) Juno cycle.
    Actually I’ve never seen so many patch sets in one cyc…

  • September 1, 2014
    OpenStack: use ephemeral and persistent root storage for different hypervisors

    {% img center http://sebastien-han.fr/images/epheremal-persistent-root-storage-different-hypervisor.jpg OpenStack: use ephemeral and persistent root storage for different hypervisors %}

    Computes with Ceph image backend and computes with local image ba…

  • August 31, 2014
    What cinder volume is missing an RBD object ?

    Although it is extremely unlikely to loose an object stored in Ceph, it is not impossible. When it happens to a Cinder volume based on RBD, knowing which has an object missing will help with disaster recovery. The list_missing command … Continue reading

  • August 25, 2014
    Ceph: mix SATA and SSD within the same box

    The use case is simple, I want to use both SSD disks and SATA disks within the same machine and ultimately create pools pointing to SSD or SATA disks.
    In order to achieve our goal, we need to modify the CRUSH map.
    My example has 2 SATA disks and 2 SS…

  • August 25, 2014
    Ceph Node.js Bindings for Librados
    1
    2
    3
    4
    5
    6
    7
    8
    9
    
    var cluster = new rados.Rados( "ceph", "client.admin", "/etc/ceph/ceph.conf");
    cluster.connect();
    
    var ioctx = new rados.Ioctx(cluster, "data");
    ioctx.aio_write("testfile2", new Buffer("1234567879ABCD"), 14, 0, function (err) {
      if (err) {
        throw err;
      }
      ...
    

    To my knowledge, there is not yet any wrapper Node.js for librados. (I guess that Alexandre has not found again the code it had begun. http://tracker.ceph.com/issues/4230). So, I started to make a draft of a plugin (when I have some time). For now I am not using it, but it allows me to discover Node. If people are interested this is here :

    https://github.com/ksperis/node-rados

    (suggestions are welcome, especially on the Error Handling, the use of libuv, buffers / strings, and everything else…)

    All is not yet implemented, but the basic functions are present.

    For example (example.js file in repo):

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    
    var rados = require('./build/Release/rados');
    
    // EXEMPLE FILE
    
    //==================================
    //     Connect to cluster
    //==================================
    var cluster = new rados.Rados( "ceph", "client.admin", "/etc/ceph/ceph.conf");
    
    var err = cluster.connect();
    if (err) {
      // On connection error
      console.log("Error " + err);
      throw err;
    }
    
    // Print cluster FSID, pools
    console.log( "fsid : " + cluster.get_fsid() );
    console.log( "ls pools : " + cluster.pool_list() );
    
    
    //==================================
    //     Create IOCTX
    //==================================
    var ioctx = new rados.Ioctx(cluster, "data");
    
    console.log(" --- RUN Sync Write / Read --- ");
    // Sync write_full
    ioctx.write_full("testfile1", new Buffer("01234567ABCDEF"));
    
    // Sync Read
    console.log( "Read data : " +
      ioctx.read("testfile1", ioctx.stat("testfile1").psize).toString() );
    
    // Remove
    ioctx.remove("testfile1");
    
    console.log(" --- RUN ASync Write / Read --- ");
    // ASync write_full
    ioctx.aio_write("testfile2", new Buffer("1234567879ABCD"), 14, 0, function (err) {
      if (err) {
        throw err;
      }
    
      ioctx.aio_read("testfile2", 14, 0, function (err, data) {
      if (err) {
        throw err;
      }
    
       console.log("[async callback] data = " + data.toString());
    
      });
    
    });
    
    
    //==================================
    //     Read / Write Attributes
    //==================================
    
    console.log(" --- RUN Attributes Write / Read --- ");
    
    ioctx.setxattr("testfile3", "attr1", "first attr");
    ioctx.setxattr("testfile3", "attr2", "second attr");
    ioctx.setxattr("testfile3", "attr3", "last attr value");
    
    var attrs = ioctx.getxattrs("testfile3");
    
    console.log("testfile3 xattr = %j", attrs);
    
    
    // destroy ioctx and close cluster after aio_flush
    ioctx.aio_flush_async(function (err) {
      ioctx.destroy();
      cluster.shutdown();
    });
    
    
    process.exit(code=0)
    
    // OTHER EXEMPLES
    
    //   Read Sync file in chunks
    var file = "testfile";
    var   fileSize = ioctx.stat(file).psize,
      chunkSize = 512,
        bytesRead = 0;
    
    
    while (bytesRead < fileSize) {
        if ((bytesRead + chunkSize) > fileSize) {
            chunkSize = (fileSize - bytesRead);
        }
        var buffer = ioctx.read(file, chunkSize, bytesRead);
        bytesRead += chunkSize;
        process.stdout.write(buffer.toString());
    }
    
    
    //   Read Async file in chunks
    var file = "testfile";
    var   fileSize = ioctx.stat(file).psize,
      chunkSize = 512,
        bytesRead = 0;
    
    
    while (bytesRead < fileSize) {
        if ((bytesRead + chunkSize) > fileSize) {
            chunkSize = (fileSize - bytesRead);
        }
        ioctx.aio_read(file, chunkSize, bytesRead, function (err, data) {
          process.stdout.write(data.toString());
        });
        bytesRead += chunkSize;
    }
    
    
    //   Use snapshot
    ioctx.write_full("testfile10", new Buffer("version1"));
    ioctx.snap_create("snaptest1");
    ioctx.write_full("testfile10", new Buffer("version2"));
    ioctx.snap_create("snaptest2");
    ioctx.write_full("testfile10", new Buffer("version3"));
    ioctx.snap_create("snaptest3");
    
    ioctx.snap_rollback("testfile10", "snaptest2");
    console.log(ioctx.read("testfile10").toString());
    
    ioctx.snap_remove("snaptest1");
    ioctx.snap_remove("snaptest2");
    ioctx.snap_remove("snaptest3");
    
  • August 24, 2014
    Update: OpenStack Summit Paris 2014 – CFS

    An update on my talk submission for the OpenStack summit this year in Paris: my speech about Ceph performance analysis was not chosen by the committee for the official agenda. But at least one piece of good news: Marc’s talk will be part of t…

  • August 20, 2014
    Ceph Primary Affinity

    This option allows you to answer a fairly constant worry in the case of heterogeneous cluster. Indeed, all HDD do not have the same performance or not the same ratio performance / size.
    With this option, it is possible to reduce the load on a specific disk without reducing the amount of data it contains.
    Furthermore, the option is easy to modify because it does not result in data migration. Only preference between primary / secondary will be modified and propagated to clients.

    Before playing with cluster options and tune crushmap, remember to verify that your client is compatible with those options.

    You must enable ‘mon osd allow primary affinity = true’ on the mons before you can adjust primary-affinity. note that older clients will no longer be able to communicate with the cluster.

    ( For client kernel module you can have a look to http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client. )

    Look if the monitor has the primary affinity option:
    # ceph –admin-daemon /var/run/ceph/ceph-mon.*.asok config show | grep ‘primary_affinity’
    “mon_osd_allow_primary_affinity”: “false”,

    Edit ceph.conf dans add in section [mon]:
    mon osd allow primary affinity = true

    Reload mon and test.
    We look how many pg is primary on osd.0, and how many is secondary :

    # ceph pg dump | grep active+clean | egrep "\[0," | wc -l
    100
    # ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
    80
    

    Try to change primary affinity :

    # ceph osd primary-affinity osd.0 0.5
    set osd.0 primary-affinity to 0.5 (8327682)
    
    # ceph pg dump | grep active+clean | egrep "\[0," | wc -l
    48
    # ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
    132
    
    # ceph osd primary-affinity osd.0 0
    set osd.0 primary-affinity to 0 (802)
    
    # ceph pg dump | grep active+clean | egrep "\[0," | wc -l
    0
    # ceph pg dump | grep active+clean | egrep ",0\]" | wc -l
    180
    

    Now there will be no more reading on this OSD.

  • August 18, 2014
    v0.84 released

    The next Ceph development release is here! This release contains several meaty items, including some MDS improvements for journaling, the ability to remove the CephFS file system (and name it), several mon cleanups with tiered pools, several OSD performance branches, a new “read forward” RADOS caching mode, a prototype Kinetic OSD backend, and various radosgw …Read more

  • August 18, 2014
    Scallable Thumbnaling Service With Thumbor and Ceph

    An example of using the python-ceph library for thumbnailing service.

    Thumbor is an opensource tool for thumbnail generation developed by globo.

    The tool allows to make a number of operations (crop, resize, filters …) directly via the URL. …

  • August 14, 2014
    Tell teuthology to use a local ceph-qa-suite directory

    By default teuthology will clone the ceph-qa-suite repository and use the tasks it contains. If tasks have been modified localy, teuthology can be instructed to use a local directory by inserting something like: suite_path: /home/loic/software/ceph/ceph-qa-suite in the teuthology job yaml … Continue reading

  • August 12, 2014
    v0.67.10 Dumpling released

    This stable update release for Dumpling includes primarily fixes for RGW, including several issues with bucket listings and a potential data corruption problem when multiple multi-part uploads race. There is also some throttling capability added in the OSD for scrub that can mitigate the performance impact on production clusters. We recommend that all Dumpling users …Read more

  • August 12, 2014
    RBD Replication

    A simple exemple to make Replication for RBD.

    Based on this post from scuttlemonkey : http://ceph.com/dev-notes/incremental-snapshots-with-rbd/, here is a sample script to synchronize rbd image on a remote cluster (eg for backups).
    In the example below, the sync is made to an “archive” pool on the same cluster.
    (For remote host, you need to use ssh key.)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    
    #!/bin/bash
    
    pool='rbd'
    pooldest='archive'
    rbd="myrbd"
    destination_host="127.0.0.1"
    snapname='rbd-sync-'
    
    
    # Retreive last synced id
    expr=" $snapname\([[:digit:]]\+\)"
    if rbd info $pool/$rbd >/dev/null 2>&1; then
            rbd snap ls $pool/$rbd | grep "$expr" | sed  "s/.*$expr.*/\1/g" | sort -n > /tmp/rbd-sync-snaplistlocal
    else
            echo "no image $pool/$rbd"
            return
    fi
    if ssh $destination_host rbd info $pooldest/$rbd >/dev/null 2>&1; then
            ssh $destination_host rbd snap ls $pooldest/$rbd | grep "$expr" | sed "s/.*$expr.*/\1/g" | sort -n > /tmp/rbd-sync-snaplistremote
    else
            echo "" > /tmp/rbd-sync-snaplistremote
    fi
    syncid=$(comm -12 /tmp/rbd-sync-snaplistlocal /tmp/rbd-sync-snaplistremote | tail -n1)
    lastid=$(cat /tmp/rbd-sync-snaplistlocal /tmp/rbd-sync-snaplistremote | sort -n | tail -n1)
    nextid=$(($lastid + 1))
    
    
    # Initial sync
    if [ "$syncid" = "" ]; then
            echo "Initial sync with id $nextid"
            rbd snap create $pool/$rbd@$snapname$nextid
            rbd export --no-progress $pool/$rbd@$snapname$nextid - \
            | ssh $destination_host rbd import --image-format 2 - $pooldest/$rbd
            ssh $destination_host rbd snap create $pooldest/$rbd@$snapname$nextid
    
    # Incremental sync
    else
            echo "Found synced id : $syncid"
            rbd snap create $pool/$rbd@$snapname$nextid
    
            echo "Sync $syncid -> $nextid"
    
            rbd export-diff --no-progress --from-snap $snapname$syncid $pool/$rbd@$snapname$nextid - \
            | ssh $destination_host rbd import-diff - $pooldest/$rbd
    fi
    

    An other exemple : https://www.rapide.nl/blog/item/ceph_-_rbd_replication

Careers