v12.2.11 Luminous released

This is the eleventh bug fix release of the Luminous v12.2.x long term stable release series. We recommend that all users upgrade to this release. Please note the following precautions while upgrading.

Notable Changes ¶

• This release fixes the pg log hard limit bug from v12.2.9 (https://tracker.ceph.com/issues/36686). A flag called pglog_hardlimit has been introduced, which is off by default. This flag enables the feature that limits the length of the pg log. Users should run ceph osd set pglog_hardlimit after completely upgrading to 12.2.11. Once the cluster has this flag set, the length of the pg log will be capped by a hard limit. Once set, the flag must not be unset anymore.
• There have been fixes to RGW dynamic and manual resharding, which no longer leaves behind stale bucket instances to be removed manually. For finding and cleaning up older instances from a reshard a radosgw-admin command reshard stale-instances list and reshard stale-instances rm should do the necessary cleanup.
• CVE-2018-14662: mon: limit caps allowed to access the config store
• CVE-2018-16846: rgw: enforce bounds on max-keys/max-uploads/max-parts issue#35994
• CVE-2018-16889: rgw: sanitize customer encryption keys from log output in v4 auth issue#37847

Changelog ¶

• build/ops: cmake: link unittest_compression against gtest (pr#24921, Willem Jan Withagen)
• build/ops: run-make-check.sh ccache tweaks (issue#24826, issue#24817, issue#24777, pr#23902, Nathan Cutler, Erwan Velu)
• ceph-bluestore-tool: fix set label functionality for specific keys (pr#25187, Igor Fedotov)
• ceph-create-keys: fix octal notation for Python 3 without losing compatibility with Python 2 (issue#37643, pr#25532, James Page)
• cephfs: ceph-volume-client: allow setting mode of CephFS volumes (pr#25407, Tom Barron)
• cephfs-journal-tool: make –rank argument mandatory (pr#24728, Venky Shankar)
• cephfs: mgr/status: fix fs status subcommand did not show standby-replay MDS’ perf info (issue#36575, issue#36399, pr#25032, Zhi Zhang)
• cephfs: race of updating wanted caps (issue#37635, issue#37464, pr#25762, “Yan, Zheng”)
• ceph-volume: Adapt code to support Python3 (pr#26030, Volker Theile)
• ceph-volume add device_id to inventory listing (pr#25350, Jan Fajerski)
• ceph-volume: enable device discards (issue#36532, pr#25748, Jonas Jelten)
• ceph-volume: fix Batch object in py3 environments (pr#25552, Jan Fajerski)
• ceph-volume: fix JSON output in inventory (issue#37390, pr#25922, Sebastian Wagner)
• ceph-volume: Fix TypeError: join() takes exactly one argument (2 given) (issue#37595, pr#25772, Sebastian Wagner)
• ceph-volume fix TypeError on dmcrypt when using Python3 (pr#26114, Alfredo Deza)
• ceph-volume: introduce class hierachy for strategies (pr#25553, Jan Fajerski, Alfredo Deza)
• ceph-volume: mark a device not available if it belongs to ceph-disk (pr#26117, Andrew Schoen)
• ceph-volume normalize comma to dot for string to int conversions (issue#37442, pr#25776, Alfredo Deza)
• ceph-volume: set permissions right before prime-osd-dir (issue#37486, pr#25778, Andrew Schoen, Alfredo Deza)
• ceph-volume tests/functional declare ceph-ansible roles instead of importing them (issue#37805, pr#25838, Alfredo Deza)
• ceph-volume zap devices associated with an OSD ID and/or OSD FSID (pr#26014, Alfredo Deza)
• ceph-volume: zap: improve zapping to remove all partitions and all LVs, encrypted or not (issue#37449, pr#25352, Alfredo Deza)
• cli: dump osd-fsid as part of osd find(issue#37966, pr#26036, Noah Watkins)
• client: do not move f->pos untill success write (issue#37631, pr#25684, Junhui Tang)
• client: explicitly show blacklisted state via asok status command (issue#36456, issue#36352, pr#24994, Jonathan Brielmaier, Zhi Zhang)
• client: fix fuse client hang because its pipe to mds is not ok4 (issue#37829, issue#36079, pr#25904, Guan yunfei)
• client: request next osdmap for blacklisted client (issue#36668, issue#36691, pr#24986, Zhi Zhang)
• common: auth/AuthSessionHandler: no handler if no session key (issue#37427, issue#36443, pr#25297, Sage Weil)
• common/blkdev, ceph-volume: improve get_device_id (pr#25752, Sage Weil)
• common: fix memory leaks in WeightedPriorityQueue (issue#37429, issue#36248, pr#25296, Radoslaw Zarzynski)
• common: (mon) command sanitization accepts floats when Int type is defined resulting in exception fault in ceph-mon (issue#26919, pr#24374, Sage Weil)
• common: shut up some warnings (pr#24648, Kefu Chai)
• config: drop config::lock when invoking config observer (issue#37762, pr#25833, Kefu Chai, Venky Shankar)
• core: bluestore: rename does not old ref to replacement onode at old name (issue#36541, issue#36638, pr#24989, Jonathan Brielmaier, Sage Weil)
• core: enable the pg deletion process to be throttled (issue#36321, pr#24501, David Zafman)
• core: mgr crash on scrub of unconnected osd (issue#36110, issue#36464, pr#25030, Sage Weil)
• core: mon osdmap cash too small during upgrade to mimic (issue#36506, pr#25021, Sage Weil)
• core: Objecter: add ignore cache flag if got redirect reply (issue#36657, pr#25074, Iain Buclaw, Jonathan Brielmaier)
• core: os/bluestore_tool: fix bluefs expand (pr#25384, Igor Fedotov)
• core: rados rm –force-full is blocked when cluster is in full status (issue#36436, pr#25018, Yang Honggang)
• crushtool: add –reclassify operation to convert legacy crush maps to use device classes (pr#25307, Sage Weil)
• debian: correct ceph-common relationship with older radosgw package (pr#24997, Matthew Vernon)
• doc: broken link on troubleshooting-mon page (pr#25500, James McClune)
• doc: fix broken fstab url in cephfs/fuse (issue#36286, pr#24434, Jos Collin)
• doc: Fix typo error on cephfs/fuse/ (issue#36180, issue#36309, pr#24752, Karun Josy)
• doc: Put command template into literal block (pr#25001, Alexey Stupnikov)
• doc/rados: update bluestore provisioning and autotuning docs (issue#37341, pr#25284, Mark Nelson)
• doc: show edit on github links and version warnings (pr#25267, Neha Ojha, Noah Watkins)
• doc/user-management: Remove obsolete reset caps command (issue#37663, pr#25609, Brad Hubbard)
• examples: fix link order in librados example Makefile (issue#37795, pr#25829, Mahati Chamarthy)
• extend reconnect period when mds is busy (issue#37739, pr#25784, “Yan, Zheng”)
• fsck: cid is improperly matched to oid (issue#36145, issue#32731, pr#24705, Sage Weil)
• libcephfs: expose CEPH_SETATTR_MTIME_NOW and CEPH_SETATTR_ATIME_NOW (issue#36206, issue#35961, pr#24465, Zhu Shangzhong)
• librbd: fix missing unblock_writes if shrink is not allowed (issue#37363, issue#36778, pr#25253, runsisi)
• librbd: reset snaps in rbd_snap_list() (issue#37535, issue#37508, pr#25458, Kefu Chai)
• mds: add “drop cache” command (issue#36695, issue#36281, pr#24468, Rishabh Dave, Patrick Donnelly, Venky Shankar)
• mds: clean up log messages for standby-replay (pr#25804, Patrick Donnelly)
• mds: create heartbeat grace config option (issue#37674, issue#37820, pr#25889, Patrick Donnelly)
• mds: directories pinned keep being replicated back and forth between exporting mds and importing mds (issue#37368, issue#37606, pr#25522, Xuehan Xu)
• mds: disallow dumping huge caches to formatter (issue#37608, pr#25567, Venky Shankar)
• mds: do not call Journaler::_trim twice (issue#37566, issue#37629, pr#25562, Tang Junhui)
• mds: fix bug filelock stuck at LOCK_XSYN leading client can’t read data (issue#37700, issue#37333, pr#25677, Guan yunfei)
• mds: fix incorrect l_pq_executing_ops statistics when meet an invalid item in purge queue (issue#37627, issue#37567, pr#25560, Junhui Tang)
• mds: fix infinite loop in OpTracker::check_ops_in_flight (issue#37977, pr#26048, “Yan, Zheng”)
• mds: fix infinite loop in OpTracker::check_ops_in_flight (issue#37977, pr#26088, “Yan, Zheng”)
• mds: fix mds damaged due to unexpected journal length (issue#36200, pr#24440, Zhi Zhang)
• mds: migrate strays part by part when shutdown mds (issue#26926, issue#32091, pr#24324, “Yan, Zheng”)
• MDSMonitor: allow beacons from stopping MDS that was laggy (issue#37737, pr#25686, Patrick Donnelly)
• mds: obsolete MDSMap option configs (issue#37540, pr#25431, Patrick Donnelly)
• mds: purge queue recovery hangs during boot if PQ journal is damaged (issue#37899, issue#37543, pr#25968, Patrick Donnelly)
• mds: PurgeQueue write error handler does not handle EBLACKLISTED (issue#37604, pr#25524, Patrick Donnelly)
• mds: rctime not set on system inode (root) at startup (issue#36221, issue#36460, pr#25043, Patrick Donnelly)
• mds: remove duplicated l_mdc_num_strays perfcounter set (issue#37633, issue#37516, pr#25682, Zhi Zhang)
• mds: severe internal fragment when decoding xattr_map from log event (issue#37399, issue#37602, pr#25520, “Yan, Zheng”)
• mds: “src/mds/MDLog.cc: 281: FAILED ceph_assert(!capped)” during max_mds thrashing (issue#36350, issue#37092, pr#25826, “Yan, Zheng”)
• mgr/balancer: add cmd to list all plans (issue#37420, pr#25259, Yang Honggang)
• mgr/balancer: add crush_compat_metrics param (issue#37413, pr#25257, Dan van der Ster)
• mgr: balancer: python 3 compat fixes (issue#37416, pr#25258, Noah Watkins)
• mgr: fix crash due to multiple sessions from daemons with same name (pr#25867, Mykola Golub)
• mgr: hold lock while accessing the request list and submitting request (pr#25047, Jerry Lee)
• mgr: Module ‘influx’ has failed (issue#25201, pr#25184, Nathan Cutler, Wido den Hollander)
• mgr: prometheus: added bluestore db and wal devices to ceph_disk_occupation metric.// (issue#37362, pr#25216, Konstantin Shalygin)
• mgr: race between daemon state and service map in ‘service status’ (issue#37478, issue#36656, pr#25369, Mykola Golub)
• mgr: [restful] deep_scrub is not a valid OSD command (issue#36720, issue#36750, pr#25041, Boris Ranto)
• mon: mark REMOVE_SNAPS messages as no_reply (issue#37568, issue#37694, pr#25779, “Yan, Zheng”)
• mon/OSDMonitor: do not populate void pg_temp into nextmap (issue#37811, pr#25845, Aleksei Zakharov)
• mon: shutdown messenger early to avoid accessing deleted logger (issue#37780, issue#37813, pr#25847, ningtao)
• os/bluestore: avoid frequent allocator dump on bluefs rebalance failure (pr#24543, Igor Fedotov)
• os/bluestore/BlueStore.cc: 1025: FAILED assert(buffer_bytes >= b->length) from ObjectStore/StoreTest.ColSplitTest2/2 (issue#26943, issue#24439, pr#24992, Jonathan Brielmaier, Sage Weil)
• os/bluestore: handle spurious read errors (issue#22464, pr#24649, Paul Emmerich)
• osd: backport recent upmap fixes (pr#25418, ningtao, xie xingguo)
• osdc/Objecter: update op_target_t::paused in _calc_target (issue#37398, issue#37553, pr#25719, Song Shun, runsisi)
• osdc: reduce ObjectCacher’s memory fragments (issue#36642, issue#36192, pr#24872, “Yan, Zheng”)
• osd: failed assert when osd_memory_target options mismatch (issue#37697, issue#37507, pr#25604, xie xingguo)
• osd/mon: pg log hard limit with upgrades fixed (issue#37903, issue#21416, pr#25949, Neha Ojha, xie xingguo)
• osd/OSD.cc: log slow requests in OSD logs (pr#25824, Neha Ojha)
• osd/OSDMap: cancel mapping if target osd is out (issue#37501, pr#25698, ningtao, xie xingguo)
• osd: potential deadlock in PG::_scan_snaps when repairing snap mapper (issue#36630, pr#24833, Mykola Golub)
• osd: Prioritize user specified scrubs (issue#37343, issue#37269, pr#25514, kungf, David Zafman)
• osd: race condition opening heartbeat connection (issue#36602, issue#36636, pr#25035, Sage Weil)
• osd: RBD client IOPS pool stats are incorrect (2x higher; includes IO hints as an op) (issue#24909, issue#36556, pr#25025, Jason Dillaman)
• pybind/mgr/status: fix ceph fs status in py3 environments (issue#37573, issue#37625, pr#25695, Jan Fajerski)
• rbd: pybind: added missing RBD_FLAG_FAST_DIFF_INVALID constant (issue#36407, pr#25006, Jason Dillaman)
• rbd: [rbd-mirror] periodic mirror status timer might fail to be scheduled (issue#36500, issue#36554, pr#24917, Nathan Cutler, Jason Dillaman)
• rgw: add ssl support to beast frontend (issue#22832, issue#24358, issue#23680, pr#24621, Casey Bodley)
• rgw: apply quota config to users created via external auth (issue#24595, issue#36222, pr#24547, Casey Bodley, Matt Benjamin)
• rgw: beast frontend fails to parse ipv6 endpoints (issue#36733, issue#36662, pr#25512, Casey Bodley)
• rgw: bucket resharding fixes (issue#37446, issue#36688, pr#25326, Orit Wasserman, Abhishek Lekshmanan, J. Eric Ivancich)
• rgw: catch exceptions from librados::NObjectIterator (issue#37091, issue#37475, pr#25289, Casey Bodley)
• rgw: Don’t treat colons specially in resource part of ARN (issue#37482, issue#23817, pr#25387, Adam C. Emerson)
• rgw: es fixes for working with nfs ganesha (issue#37349, issue#36233, issue#22758, pr#25444, Abhishek Lekshmanan)
• rgw_file: user info never synced since librgw init (issue#37549, pr#25484, Tao Chen)
• rgw: fixes for zone deletion (issue#37328, issue#37466, pr#25320, Abhishek Lekshmanan)
• rgw: fix max-size in radosgw-admin and REST Admin API (issue#37519, pr#25448, Nick Erdmann)
• rgw: fix version bucket stats (issue#37563, issue#21429, pr#25644, Shasha Lu)
• rgw: librgw: crashes in multisite configuration (issue#36302, issue#36414, pr#24909, Casey Bodley)
• rgw: multisite: sync gets stuck retrying deletes that fail with ERR_PRECONDITION_FAILED (issue#37551, issue#37448, pr#25506, Casey Bodley)
• rgw: radosgw-admin: translate reshard status codes (trivial) (issue#37284, issue#36486, pr#25195, Matt Benjamin)
• rgw: rgw-admin: reshard add can add a non-existent bucket (issue#36449, issue#36757, pr#25088, Jonathan Brielmaier, Abhishek Lekshmanan)
• rgw: SSE encryption does not detect ssl termination in proxy (issue#36644, issue#27221, pr#24944, Jonathan Brielmaier, Casey Bodley)
• rpm: Use hardened LDFLAGS (issue#36316, issue#36391, pr#25173, Boris Ranto)