v10.0.3 released

sage

This is the fourth development release for Jewel. Several big pieces have been added this release, including BlueStore (a new backend for OSD to replace FileStore), many ceph-disk fixes, a new CRUSH tunable that improves mapping stability, a new librados object enumeration API, and a whole slew of OSD and RADOS optimizations.

Note that, due to general developer busyness, we aren’t building official release packages for this dev release. You can fetch autobuilt gitbuilder packages from the usual location (http://gitbuilder.ceph.com).

NOTABLE CHANGES

  • bluestore: latest and greatest (issue#14210, issue#13801, pr#6896, xie.xingguo, Jianpeng Ma, YiQiang Chen, Sage Weil, Ning Yao)
  • buffer: fix internal iterator invalidation on rebuild, get_contiguous (pr#6962, Sage Weil)
  • build: fix a few warnings (pr#6847, Orit Wasserman)
  • build: misc make check fixes (pr#7153, Sage Weil)
  • ceph-detect-init: fix py3 test (pr#7025, Kefu Chai)
  • ceph-disk: add -f flag for btrfs mkfs (pr#7222, Darrell Enns)
  • ceph-disk: ceph-disk list fails on /dev/cciss!c0d0 (issue#13970, issue#14233, issue#14230, pr#6879, Loic Dachary)
  • ceph-disk: fix failures when preparing disks with udev > 214 (issue#14080, issue#14094, pr#6926, Loic Dachary, Ilya Dryomov)
  • ceph-disk: Fix trivial typo (pr#7472, Brad Hubbard)
  • ceph-disk: warn for prepare partitions with bad GUIDs (issue#13943, pr#6760, David Disseldorp)
  • ceph-fuse: fix double decreasing the count to trim caps (issue#14319, pr#7229, Zhi Zhang)
  • ceph-fuse: fix double free of args (pr#7015, Ilya Shipitsin)
  • ceph-fuse: fix fsync() (pr#6388, Yan, Zheng)
  • ceph-fuse:print usage information when no parameter specified (pr#6868, Bo Cai)
  • ceph: improve the error message (issue#11101, pr#7106, Kefu Chai)
  • ceph.in: avoid a broken pipe error when use ceph command (issue#14354, pr#7212, Bo Cai)
  • ceph.spec.in: add copyright notice (issue#14694, pr#7569, Nathan Cutler)
  • ceph.spec.in: add license declaration (pr#7574, Nathan Cutler)
  • ceph_test_libcephfs: tolerate duplicated entries in readdir (issue#14377, pr#7246, Yan, Zheng)
  • client: check if Fh is readable when processing a read (issue#11517, pr#7209, Yan, Zheng)
  • client: properly trim unlinked inode (issue#13903, pr#7297, Yan, Zheng)
  • cls_rbd: add guards for error cases (issue#14316, issue#14317, pr#7165, xie xingguo)
  • cls_rbd: enable object map checksums for object_map_save (issue#14280, pr#7149, Douglas Fuller)
  • cmake: Add ENABLE_GIT_VERSION to avoid rebuilding (pr#7171, Kefu Chai)
  • cmake: add missing check for HAVE_EXECINFO_H (pr#7270, Casey Bodley)
  • cmake: cleanups and more features from automake (pr#7103, Casey Bodley, Ali Maredia)
  • cmake: detect bzip2 and lz4 (pr#7126, Kefu Chai)
  • cmake: fix build with bluestore (pr#7099, John Spray)
  • cmake: fix the build on trusty (pr#7249, Kefu Chai)
  • cmake: made rocksdb an imported library (pr#7131, Ali Maredia)
  • cmake: no need to run configure from run-cmake-check.sh (pr#6959, Orit Wasserman)
  • cmake: test_build_libcephfs needs ${ALLOC_LIBS} (pr#7300, Ali Maredia)
  • common/address_help.cc: fix the leak in entity_addr_from_url() (issue#14132, pr#6987, Qiankun Zheng)
  • common: add thread names (pr#5882, Igor Podoski)
  • common: assert: abort() rather than throw (pr#6804, Adam C. Emerson)
  • common: buffer/assert minor fixes (pr#6990, Matt Benjamin)
  • common/Formatter: avoid newline if there is no output (pr#5351, Aran85)
  • common: improve shared_cache and simple_cache efficiency with hash table (pr#6909, Ning Yao)
  • common/lockdep: increase max lock names (pr#6961, Sage Weil)
  • common: new timekeeping common code, and Objecter conversion (pr#5782, Adam C. Emerson)
  • common: signal_handler: added support for using reentrant strsignal() implementations vs. sys_siglist[] (pr#6796, John Coyle)
  • config: complains when a setting is not tracked (issue#11692, pr#7085, Kefu Chai)
  • configure: detect bz2 and lz4 (issue#13850, issue#13981, pr#7030, Kefu Chai)
  • correct radosgw-admin command (pr#7006, YankunLi)
  • crush: add chooseleaf_stable tunable (pr#6572, Sangdi Xu, Sage Weil)
  • crush: clean up whitespace removal (issue#14302, pr#7157, songbaisen)
  • crush/CrushTester: check for overlapped rules (pr#7139, Kefu Chai)
  • crushtool: improve usage/tip messages (pr#7142, xie xingguo)
  • crushtool: set type 0 name “device” for –build option (pr#6824, Sangdi Xu)
  • doc: adding “–allow-shrink” in decreasing the size of the rbd block to distinguish from the increasing option (pr#7020, Yehua)
  • doc: admin/build-doc: make paths absolute (pr#7119, Dan Mick)
  • doc: dev: document ceph-qa-suite (pr#6955, Loic Dachary)
  • doc: document “readforward” and “readproxy” cache mode (pr#7023, Kefu Chai)
  • doc: fix “mon osd down out subtree limit” option name (pr#7164, François Lafont)
  • doc: fix typo (pr#7004, tianqing)
  • doc: Updated the rados command man page to include the –run-name opt… (issue#12899, pr#5900, ritz303)
  • fs: be more careful about the “mds setmap” command to prevent breakage (issue#14380, pr#7262, Yan, Zheng)
  • helgrind: additional race conditionslibrbd: journal replay should honor inter-event dependencies (pr#7274, Jason Dillaman)
  • helgrind: fix real (and imaginary) race conditions (issue#14163, pr#7208, Jason Dillaman)
  • kv: implement value_as_ptr() and use it in .get() (pr#7052, Piotr Dałek)
  • librados: add c++ style osd/pg command interface (pr#6893, Yunchuan Wen)
  • librados: fix several flaws introduced by the enumeration_objects API (issue#14299, issue#14301, issue#14300, pr#7156, xie xingguo)
  • librados: new style (sharded) object listing (pr#6405, John Spray, Sage Weil)
  • librados: potential null pointer access in list_(n)objects (issue#13822, pr#6639, xie xingguo)
  • librbd: exit if parent’s snap is gone during clone (issue#14118, pr#6968, xie xingguo)
  • librbd: fix potential memory leak (issue#14332, issue#14333, pr#7174, xie xingguo)
  • librbd: fix snap_exists API return code overflow (issue#14129, pr#6986, xie xingguo)
  • librbd: journal replay should honor inter-event dependencies (pr#7019, Jason Dillaman)
  • librbd: return error if we fail to delete object_map head object (issue#14098, pr#6958, xie xingguo)
  • librbd: small fixes for error messages and readahead counter (issue#14127, pr#6983, xie xingguo)
  • librbd: uninitialized state in snap remove state machine (pr#6982, Jason Dillaman)
  • mailmap: hange organization for Dongmao Zhang (pr#7173, Dongmao Zhang)
  • mailmap: Igor Podoski affiliation (pr#7219, Igor Podoski)
  • mailmap update (pr#7210, M Ranga Swami Reddy)
  • mailmap updates (pr#6992, Loic Dachary)
  • mailmap updates (pr#7189, Loic Dachary)
  • man: document listwatchers cmd in “rados” manpage (pr#7021, Kefu Chai)
  • mds: advance clientreplay when replying (issue#14357, pr#7216, John Spray)
  • mds: expose state of recovery to status ASOK command (issue#14146, pr#7068, Yan, Zheng)
  • mds: fix client cap/message replay order on restart (issue#14254, issue#13546, pr#7199, Yan, Zheng)
  • mds: fix standby replay thread creation (issue#14144, pr#7132, John Spray)
  • mds: we should wait messenger when MDSDaemon suicide (pr#6996, Wei Feng)
  • mon: add osd blacklist clear (pr#6945, John Spray)
  • mon: add RAW USED column to ceph df detail (pr#7087, Ruifeng Yang)
  • mon: degrade a log message to level 2 (pr#6929, Kongming Wu)
  • mon: fix coding-style on PG related Monitor files (pr#6881, Wido den Hollander)
  • mon: fixes related to mondbstore->get() changes (pr#6564, Piotr Dałek)
  • mon: fix reuse of osd ids (clear osd info on osd deletion) (issue#13988, pr#6900, Loic Dachary, Sage Weil)
  • mon: fix the can’t change subscribe level bug in monitoring log (pr#7031, Zhiqiang Wang)
  • mon/MDSMonitor: add confirmation to “ceph mds rmfailed” (issue#14379, pr#7248, Yan, Zheng)
  • mon: modify a dout level in OSDMonitor.cc (pr#6928, Yongqiang He)
  • mon: MonmapMonitor: don’t expose uncommitted state to client (pr#6854, Joao Eduardo Luis)
  • mon/OSDMonitor: osdmap laggy set a maximum limit for interval (pr#7109, Zengran Zhang)
  • mon: paxos is_recovering calc error (pr#7227, Weijun Duan)
  • mon/PGMap: show rd/wr iops separately in status reports (pr#7072, Cilang Zhao)
  • mon: PGMonitor: acting primary diff with cur_stat, should not set pg to stale (pr#7083, Xiaowei Chen)
  • msg: add override to virutal methods (pr#6977, Michal Jarzabek)
  • msg/async: cleanup dead connection and misc things (pr#7158, Haomai Wang)
  • msg/async: don’t use shared_ptr to manage EventCallback (pr#7028, Haomai Wang)
  • msg: filter out lo addr when bind osd addr (pr#7012, Ji Chen)
  • msg: removed unneeded includes from Dispatcher (pr#6814, Michal Jarzabek)
  • msg: remove unneeded inline (pr#6989, Michal Jarzabek)
  • msgr: fix large message data content length causing overflow (pr#6809, Jun Huang, Haomai Wang)
  • msg/simple: pipe: memory leak when signature check failed (pr#7096, Ruifeng Yang)
  • msg/simple: remove unneeded friend declarations (pr#6924, Michal Jarzabek)
  • objecter: avoid recursive lock of Objecter::rwlock (pr#7343, Yan, Zheng)
  • os/bluestore: fix bluestore_wal_transaction_t encoding test (pr#7419, Kefu Chai, Brad Hubbard)
  • osd: add cache hint when pushing raw clone during recovery (pr#7069, Zhiqiang Wang)
  • osd: avoid debug std::string initialization in PG::get/put (pr#7117, Evgeniy Firsov)
  • osd: avoid osd_op_thread suicide because osd_scrub_sleep (pr#7009, Jianpeng Ma)
  • osd: bluestore: bluefs: fix several small bugs (issue#14344, issue#14343, pr#7200, xie xingguo)
  • osd: bluestore: don’t include when building without libaio (issue#14207, pr#7169, Mykola Golub)
  • osd: bluestore: fix bluestore onode_t attr leak (pr#7125, Ning Yao)
  • osd: bluestore: fix bluestore_wal_transaction_t encoding test (pr#7168, Kefu Chai)
  • osd: bluestore: fix several bugs (issue#14259, issue#14353, issue#14260, issue#14261, pr#7122, xie xingguo)
  • osd: bluestore: fix space rebalancing, collection split, buffered reads (pr#7196, Sage Weil)
  • osd: bluestore: more fixes (pr#7130, Sage Weil)
  • osd: cache tier: add config option for eviction check list size (pr#6997, Yuan Zhou)
  • osdc: Fix race condition with tick_event and shutdown (issue#14256, pr#7151, Adam C. Emerson)
  • osd: check health state before pre_booting (issue#14181, pr#7053, Xiaoxi Chen)
  • osd: clear pg_stat_queue after stopping pgs (issue#14212, pr#7091, Sage Weil)
  • osd: delay populating in-memory PG log hashmaps (pr#6425, Piotr Dałek)
  • osd: disable filestore_xfs_extsize by default (issue#14397, pr#7265, Ken Dreyer)
  • osd: do not keep ref of old osdmap in pg (issue#13990, pr#7007, Kefu Chai)
  • osd: drop deprecated removal pg type (pr#6970, Igor Podoski)
  • osd: FileJournal: fix return code of create method (issue#14134, pr#6988, xie xingguo)
  • osd: FileJournal: support batch peak and pop from writeq (pr#6701, Xinze Chi)
  • osd: FileStore: conditional collection of drive metadata (pr#6956, Somnath Roy)
  • osd: FileStore:: optimize lfn_unlink (pr#6649, Jianpeng Ma)
  • osd: fix null pointer access and race condition (issue#14072, pr#6916, xie xingguo)
  • osd: fix scrub start hobject (pr#7467, Sage Weil)
  • osd: fix sparse-read result code checking logic (issue#14151, pr#7016, xie xingguo)
  • osd: fix temp object removal after upgrade (issue#13862, pr#6976, David Zafman)
  • osd: fix wip (l_osd_op_wip) perf counter and remove repop_map (pr#7077, Xinze Chi)
  • osd: fix wrongly placed assert and some cleanups (pr#6766, xiexingguo, xie xingguo)
  • osd: KeyValueStore: fix return code of mkfs (pr#7036, xie xingguo)
  • osd: KeyValueStore: fix wrongly placed assert (issue#14176, issue#14178, pr#7047, xie xingguo)
  • osd: kstore: several small fixes (issue#14351, issue#14352, pr#7213, xie xingguo)
  • osd: kstore: small fixes to kstore (issue#14204, pr#7095, xie xingguo)
  • osd: make list_missing query missing_loc.needs_recovery_map (pr#6298, Guang Yang)
  • osdmap: remove unused local variables (pr#6864, luo kexue)
  • osd: memstore: fix two bugs (pr#6963, Casey Bodley, Sage Weil)
  • osd: misc FileStore fixes (issue#14192, issue#14188, issue#14194, issue#14187, issue#14186, pr#7059, xie xingguo)
  • osd: misc optimization for map utilization (pr#6950, Ning Yao)
  • osd,mon: log leveldb and rocksdb to ceph log (pr#6921, Sage Weil)
  • osd: Omap small bugs adapted (pr#6669, Jianpeng Ma, David Zafman)
  • osd: optimize the session_handle_reset function (issue#14182, pr#7054, songbaisen)
  • osd: OSDService: Fix typo in osdmap comment (pr#7275, Brad Hubbard)
  • osd: os: skip checking pg_meta object existance in FileStore (pr#6870, Ning Yao)
  • osd: PGLog: clean up read_log (pr#7092, Jie Wang)
  • osd: prevent osd_recovery_sleep from causing recovery-thread suicide (pr#7065, Jianpeng Ma)
  • osd: reduce string use in coll_t::calc_str() (pr#6505, Igor Podoski)
  • osd: release related sources when scrub is interrupted (pr#6744, Jianpeng Ma)
  • osd: remove unused OSDMap::set_weightf() (issue#14369, pr#7231, huanwen ren)
  • osd: ReplicatedPG: clean up unused function (pr#7211, Xiaowei Chen)
  • osd/ReplicatedPG: fix promotion recency logic (issue#14320, pr#6702, Sage Weil)
  • osd: several small cleanups (pr#7055, xie xingguo)
  • osd: shut down if we flap too many times in a short period (pr#6708, Xiaoxi Chen)
  • osd: skip promote for writefull w/ FADVISE_DONTNEED/NOCACHE (pr#7010, Jianpeng Ma)
  • osd: small fixes to memstore (issue#14228, issue#14229, issue#14227, pr#7107, xie xingguo)
  • osd: try evicting after flushing is done (pr#5630, Zhiqiang Wang)
  • osd: use atomic to generate ceph_tid (pr#7017, Evgeniy Firsov)
  • osd: use optimized is_zero in object_stat_sum_t.is_zero() (pr#7203, Piotr Dałek)
  • osd: utime_t, eversion_t, osd_stat_sum_t encoding optimization (pr#6902, Xinze Chi)
  • pybind: add ceph_volume_client interface for Manila and similar frameworks (pr#6205, John Spray)
  • pybind: fix build failure, remove extraneous semicolon in method (issue#14371, pr#7235, Abhishek Lekshmanan)
  • pybind/test_rbd: fix test_create_defaults (issue#14279, pr#7155, Josh Durgin)
  • qa: disable rbd/qemu-iotests test case 055 on RHEL/CentOSlibrbd: journal replay should honor inter-event dependencies (issue#14385, pr#7272, Jason Dillaman)
  • qa/workunits: merge_diff shouldn’t attempt to use striping (issue#14165, pr#7041, Jason Dillaman)
  • qa/workunits/snaps: move snap tests into fs sub-directory (pr#6496, Yan, Zheng)
  • rados: implement rm –force option to force remove when full (pr#6202, Xiaowei Chen)
  • rbd: additional validation for striping parameters (pr#6914, Na Xie)
  • rbd: add pool name to disambiguate rbd admin socket commands (pr#6904, wuxiangwei)
  • rbd: correct an output string for merge-diff (pr#7046, Kongming Wu)
  • rbd: fix static initialization ordering issues (pr#6978, Mykola Golub)
  • rbd-fuse: image name can not include snap name (pr#7044, Yongqiang He)
  • rbd-fuse: implement mv operation (pr#6938, wuxiangwei)
  • rbd: must specify both of stripe-unit and stripe-count when specifying stripingv2 feature (pr#7026, Donghai Xu)
  • rbd-nbd: add copyright (pr#7166, Li Wang)
  • rbd-nbd: fix up return code handling (pr#7215, Mykola Golub)
  • rbd-nbd: small improvements in logging and forking (pr#7127, Mykola Golub)
  • rbd: rbd order will be place in 22, when set to 0 in the config_opt (issue#14139, issue#14047, pr#6886, huanwen ren)
  • rbd: striping parameters should support 64bit integers (pr#6942, Na Xie)
  • rbd: use default order from configuration when not specified (pr#6965, Yunchuan Wen)
  • rgw: add a method to purge all associate keys when removing a subuser (issue#12890, pr#6002, Sangdi Xu)
  • rgw: add missing error code for admin op API (pr#7037, Dunrong Huang)
  • rgw: add support for “end_marker” parameter for GET on Swift account. (issue#10682, pr#4216, Radoslaw Zarzynski)
  • rgw_admin: orphans finish segfaults (pr#6652, Igor Fedotov)
  • rgw: content length (issue#13582, pr#6975, Yehuda Sadeh)
  • rgw: delete default zone (pr#7005, YankunLi)
  • rgw: do not abort radowgw server when using admin op API with bad parameters (issue#14190, issue#14191, pr#7063, Dunrong Huang)
  • rgw: Drop a debugging message (pr#7280, Pete Zaitcev)
  • rgw: fix a typo in init-radosgw (pr#6817, Zhi Zhang)
  • rgw: fix compilation warning (pr#7160, Yehuda Sadeh)
  • rgw: fix wrong check for parse() return (pr#6797, Dunrong Huang)
  • rgw: let radosgw-admin bucket stats return a standard josn (pr#7029, Ruifeng Yang)
  • rgw: modify command stucking when operating radosgw-admin metadata list user (pr#7032, Peiyang Liu)
  • rgw: modify documents and help infos’ descriptions to the usage of option date when executing command “log show” (pr#6080, Kongming Wu)
  • rgw: Parse –subuser better (pr#7279, Pete Zaitcev)
  • rgw: radosgw-admin bucket check –fix not work (pr#7093, Weijun Duan)
  • rgw: warn on suspicious civetweb frontend parameters (pr#6944, Matt Benjamin)
  • rocksdb: remove rdb sources from dist tarball (issue#13554, pr#7105, Venky Shankar)
  • stringify outputted error code and fix unmatched parentheses. (pr#6998, xie.xingguo, xie xingguo)
  • test/librbd/fsx: Use c++11 std::mt19937 generator instead of random_r() (pr#6332, John Coyle)
  • test/mon/osd-erasure-code-profile: pick new mon port (pr#7161, Sage Weil)
  • tests: add const for ec test (pr#6911, Michal Jarzabek)
  • tests: configure with rocksdb by default (issue#14220, pr#7100, Loic Dachary)
  • tests: Fix for make check. (pr#7102, David Zafman)
  • tests: notification slave needs to wait for master (issue#13810, pr#7220, Jason Dillaman)
  • tests: snap rename and rebuild object map in client update test (pr#7224, Jason Dillaman)
  • tests: unittest_bufferlist: fix hexdump test (pr#7152, Sage Weil)
  • tests: unittest_ipaddr: fix segv (pr#7154, Sage Weil)
  • tools: ceph_monstore_tool: add inflate-pgmap command (issue#14217, pr#7097, Kefu Chai)
  • tools: monstore: add ‘show-versions’ command. (pr#7073, Cilang Zhao)

GETTING CEPH