RocksDB release 9.9.3

9.9.3 (12/03/2024)

Performance Improvements

• In buffered IO mode, try to align writes on power of 2 if checksum handoff is not enabled for the file type being written.

9.9.2 (11/22/2024)

Bug Fixes

• Honor ConfigOptions.ignore_unknown_options in ParseStruct()

9.9.1 (11/30/2024)

Behavior Changes

• Updates the hidden hook RocksDbThreadYieldAndCheckAbort() to support MySQL to abort long-running query.

9.9.0 (11/18/2024)

New Features

• Multi-Column-Family-Iterator (CoalescingIterator/AttributeGroupIterator) is no longer marked as experimental
• Adds a new table property "rocksdb.newest.key.time" which records the unix timestamp of the newest key. Uses this table property for FIFO TTL and temperature change compaction.

Public API Changes

• Added a new API Transaction::GetAttributeGroupIterator that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.
• Added a new API Transaction::GetCoalescingIterator that can be used to create a multi-column-family coalescing iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.

Behavior Changes

• BaseDeltaIterator now honors the read option allow_unprepared_value.

Bug Fixes

• BaseDeltaIterator now calls PrepareValue on the base iterator in case it has been created with the allow_unprepared_value read option set. Earlier, such base iterators could lead to incorrect values being exposed from BaseDeltaIterator.
• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.
• Fix missing cases of corruption retry during DB open and read API processing.
• Fix a bug for transaction db with 2pc where an old WAL may be retained longer than needed (#13127).
• Fix leaks of some open SST files (until DB::Close()) that are written but never become live due to various failures. (We now have a check for such leaks with no outstanding issues.)
• Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions.

Performance Improvements

• Fix regression in issue #12038 due to Options::compaction_readahead_size greater than max_sectors_kb (i.e, largest I/O size that the OS issues to a block device defined in linux)

RocksDB 9.8.4

9.8.4 (11/18/2024)

Behavior Changes

• When Remote Compaction is enabled, do not purge OPTIONS file immediately by DeleteObsoleteOptionsFiles() after SetOptions(). Rely on PurgeObsoleteFiles() to clean up obsolete OPTIONS file after each compaction.

9.8.3 (11/12/2024)

Bug Fixes

• Fix missing cases of corruption retry during DB open and read API processing.

9.8.2 (11/06/2024)

Public API Changes

• Added a new API Transaction::GetAttributeGroupIterator that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.

Behavior Changes

• BaseDeltaIterator now honors the read option allow_unprepared_value.

Bug Fixes

• BaseDeltaIterator now calls PrepareValue on the base iterator in case it has been created with the allow_unprepared_value read option set. Earlier, such base iterators could lead to incorrect values being exposed from BaseDeltaIterator.
• Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions.

9.8.1 (10/31/2024)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version
  9.4.0.

9.8.0 (10/25/2024)

New Features

• All non-block_cache options in BlockBasedTableOptions are now mutable with DB::SetOptions().
  See also Bug Fixes below.
• When using iterators with BlobDB, it is now possible to load large values on an on-demand basis, i
  .e. only if they are actually needed by the application. This can save I/O in use cases where the va
  lues associated with certain keys are not needed. For more details, see the new read option allow_u nprepared_value and the iterator API PrepareValue.
• Add a new file ingestion option IngestExternalFileOptions::fill_cache to support not adding bloc
  ks from ingested files into block cache during file ingestion.
• The option allow_unprepared_value is now also supported for multi-column-family iterators (i.e.
  CoalescingIterator and AttributeGroupIterator).
• When a file with just one range deletion (standalone range deletion file) is ingested via bulk loa
  ding, it will be marked for compaction. During compaction, this type of files can be used to directl
  y filter out some input files that are not protected by any snapshots and completely deleted by the
  standalone range deletion file.

Behavior Changes

• During file ingestion, overlapping files level assignment are done in multiple batches, so that th
  ey can potentially be assigned to lower levels other than always land on L0.
• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the
  primary host. A similar technique as how we are preserving new SST files from getting purged is used
  for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.
• Trim readahead_size during scans so data blocks containing keys that are not in the same prefix as
  the seek key in Seek() are not prefetched when ReadOptions::auto_readahead_size=true (default v
  alue) and ReadOptions::prefix_same_as_start = true
• Assigning levels for external files are done in the same way for universal compaction and leveled
  compaction. The old behavior tends to assign files to L0 while the new behavior will assign the file
  s to the lowest level possible.

Bug Fixes

• Fix a longstanding race condition in SetOptions for block_based_table_factory options. The fix h
  as some subtle behavior changes because of copying and replacing the TableFactory on a change with S
  etOptions, including requiring an Iterator::Refresh() for an existing Iterator to use the latest opt
  ions.
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the com
  pressed block size rather than the actual memory allocated, which could be larger due to internal fr
  agmentation.
• GetApproximateMemTableStats() could return disastrously bad estimates 5-25% of the time. The fun
  ction has been re-engineered to return much better estimates with similar CPU cost.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option
  is kVolatileTier.
• Fix an issue in level compaction where a small CF with small compaction debt can cause the DB to a
  llow parallel compactions. (#13054)
• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as w
  ell. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_ identity_file, prefix_seek_opt_in_only

RocksDB 9.7.4

9.7.4 (10/31/2024)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.

9.7.3 (10/16/2024)

Behavior Changes

• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.

9.7.2 (10/08/2024)

Bug Fixes

• Fix a bug for surfacing write unix time: Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.

9.7.1 (09/26/2024)

Bug Fixes

• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as well. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_identity_file, prefix_seek_opt_in_only
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the compressed block size rather than the actual memory allocated, which could be larger due to internal fragmentation.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option is kVolatileTier.

9.7.0 (09/20/2024)

New Features

• Make Cache a customizable class that can be instantiated by the object registry.
• Add new option prefix_seek_opt_in_only that makes iterators generally safer when you might set a prefix_extractor. When prefix_seek_opt_in_only=true, which is expected to be the future default, prefix seek is only used when prefix_same_as_start or auto_prefix_mode are set. Also, prefix_same_as_start and auto_prefix_mode now allow prefix filtering even with total_order_seek=true.
• Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion.

Behavior Changes

• Changed the semantics of the BlobDB configuration option blob_garbage_collection_force_threshold to define
  a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (accordin
  g to blob_garbage_collection_age_cutoff). This can provide better control over space amplification at the cos
  t of slightly higher write amplification.
• Set write_dbid_to_manifest=true by default. This means DB ID will now be preserved through backups, checkpo
  ints, etc. by default. Also add write_identity_file option which can be set to false for anticipated future b
  ehavior.
• In FIFO compaction, compactions for changing file temperature (configured by option file_temperature_age_thr esholds) will compact one file at a time, instead of merging multiple eligible file together (#13018).
• Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files a
  nd IngestExternalFileOptions::allow_db_generated_files.
• Add a new file ingestion option IngestExternalFileOptions::link_files to hard link input files and preserve
  original files links after ingestion.
• DB::Close now untracks files in SstFileManager, making avaialble any space used
  by them. Prior to this change they would be orphaned until the DB is re-opened.

Bug Fixes

• Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only h
  appen when users configure CompactRangeOptions::change_level to true and the change level step of manual compac
  tion fails (#13009).
• Fix handling of dynamic change of prefix_extractor with memtable prefix filter. Previously, prefix seek cou
  ld mix different prefix interpretations between memtable and SST files. Now the latest prefix_extractor at th
  e time of iterator creation or refresh is respected.
• Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsisten
  t (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disa
  bles auto error recovery from these errors.

RocksDB 9.6.2

9.6.2 (10/31/2024)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.

9.6.1 (08/24/2024)

Bug Fixes

• Fix correctness of MultiGet across column families with user timestamp.

9.6.0 (08/19/2024)

New Features

• *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
• New option BlockBasedTableOptions::decouple_partitioned_filters should improve efficiency in serving read queries because filter and index partitions can consistently target the configured metadata_block_size. This option is currently opt-in.
• Introduce a new mutable CF option paranoid_memory_checks. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch
• Adds optional installation callback function for remote compaction

Behavior Changes

• There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
• Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
• Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
• Fixed a data race involving the background error status in unordered_write mode.
• *Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest
  file. #12882
• Fix a bug where per kv checksum corruption may be ignored in MultiGet().
• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to
  be registered simultaneously, resulting in a crash or other unpredictable behavior.

RocksDB 9.7.3

9.7.3 (2024-10-16)

Behavior Changes

• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.

9.7.2 (2024-08-10)

Bug Fixes

• Fix a bug for surfacing write unix time: Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.

9.7.1 (2024-09-26)

Bug Fixes

• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as well. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_identity_file, prefix_seek_opt_in_only
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the compressed block size rather than the actual memory allocated, which could be larger due to internal fragmentation.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option is kVolatileTier.

9.7.0 (2024-09-20)

New Features

• Make Cache a customizable class that can be instantiated by the object registry.
• Add new option prefix_seek_opt_in_only that makes iterators generally safer when you might set a prefix_extractor. When prefix_seek_opt_in_only=true, which is expected to be the future default, prefix seek is only used when prefix_same_as_start or auto_prefix_mode are set. Also, prefix_same_as_start and auto_prefix_mode now allow prefix filtering even with total_order_seek=true.
• Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion.

Behavior Changes

• Changed the semantics of the BlobDB configuration option blob_garbage_collection_force_threshold to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to blob_garbage_collection_age_cutoff). This can provide better control over space amplification at the cost of slightly higher write amplification.
• Set write_dbid_to_manifest=true by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add write_identity_file option which can be set to false for anticipated future behavior.
• In FIFO compaction, compactions for changing file temperature (configured by option file_temperature_age_thresholds) will compact one file at a time, instead of merging multiple eligible file together (#13018).
• Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files.
• Add a new file ingestion option IngestExternalFileOptions::link_files to hard link input files and preserve original files links after ingestion.
• DB::Close now untracks files in SstFileManager, making avaialble any space used
  by them. Prior to this change they would be orphaned until the DB is re-opened.

Bug Fixes

• Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#13009).
• Fix handling of dynamic change of prefix_extractor with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest prefix_extractor at the time of iterator creation or refresh is respected.
• Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors.

RocksDB 9.6.1

9.6.1 (2024-08-24)

Bug Fixes

• Fix correctness of MultiGet across column families with user timestamp.

9.6.0 (2024-08-19)

New Features

• *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
• New option BlockBasedTableOptions::decouple_partitioned_filters should improve efficiency in serving read queries because filter and index partitions can consistently target the configured metadata_block_size. This option is currently opt-in.
• Introduce a new mutable CF option paranoid_memory_checks. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch
• Adds optional installation callback function for remote compaction

Behavior Changes

• There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
• Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
• Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
• Fixed a data race involving the background error status in unordered_write mode.
• *Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest file. #12882
• Fix a bug where per kv checksum corruption may be ignored in MultiGet().
• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.

RocksDB 9.5.2

9.5.2 (2024-08-13)

Bug Fixes

• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch

9.5.1 (2024-08-02)

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.

9.5.0 (2024-07-19)

Public API Changes

• Introduced new C API function rocksdb_writebatch_iterate_cf for column family-aware iteration over the contents of a WriteBatch
• Add support to ingest SST files generated by a DB instead of SstFileWriter. This can be enabled with experimental option IngestExternalFileOptions::allow_db_generated_files.

Behavior Changes

• When calculating total log size for the log_size_for_flush argument in CreateCheckpoint API, the size of the archived log will not be included to avoid unnecessary flush

Bug Fixes

• Fix a major bug in which an iterator using prefix filtering and SeekForPrev might miss data when the DB is using whole_key_filtering=false and partition_filters=true.
• Fixed a bug where OnErrorRecoveryBegin() is not called before auto recovery starts.
• Fixed a bug where event listener reads ErrorHandler's bg_error_ member without holding db mutex(#12803).
• Fixed a bug in handling MANIFEST write error that caused the latest valid MANIFEST file to get deleted, resulting in the DB being unopenable.
• Fixed a race between error recovery due to manifest sync or write failure and external SST file ingestion. Both attempt to write a new manifest file, which causes an assertion failure.

Performance Improvements

• Fix an issue where compactions were opening table files and reading table properties while holding db mutex_.
• Reduce unnecessary filesystem queries and DB mutex acquires in creating backups and checkpoints.

RocksDB 9.4.0

9.4.0 (2024-06-23)

New Features

• Added a CompactForTieringCollectorFactory to auto trigger compaction for tiering use case.
• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the GetEntityForUpdate API.
• Added a new "count" command to the ldb repl shell. By default, it prints a count of keys in the database from start to end. The options --from= and/or --to= can be specified to limit the range.
• Add rocksdb_writebatch_update_timestamps, rocksdb_writebatch_wi_update_timestamps in C API.
• Add rocksdb_iter_refresh in C API.
• Add rocksdb_writebatch_create_with_params, rocksdb_writebatch_wi_create_with_params to create WB and WBWI with all options in C API

Public API Changes

• Deprecated names LogFile and VectorLogPtr in favor of new names WalFile and VectorWalPtr.
• Introduce a new universal compaction option CompactionOptionsUniversal::max_read_amp which allows user to define the limit on the number of sorted runs separately from the trigger for compaction (level0_file_num_compaction_trigger) #12477.

Behavior Changes

• Inactive WALs are immediately closed upon being fully sync-ed rather than in a background thread. This is to ensure LinkFile() is not called on files still open for write, which might not be supported by some FileSystem implementations. This should not be a performance issue, but an opt-out is available with with new DB option background_close_inactive_wals.

Bug Fixes

• Fix a rare case in which a hard-linked WAL in a Checkpoint is not fully synced (so might lose data on power loss).
• Fixed the output of the ldb dump_wal command for PutEntity records so it prints the key and correctly resets the hexadecimal formatting flag after printing the wide-column entity.
• Fixed an issue where PutEntity records were handled incorrectly while rebuilding transactions during recovery.
• Various read operations could ignore various ReadOptions that might be relevant. Fixed many such cases, which can result in behavior change but a better reflection of specified options.

Performance Improvements

• Improved write throughput to memtable when there's a large number of concurrent writers and allow_concurrent_memtable_write=true(#12545)

RocksDB 9.3.1

9.3.1 (2024-05-25)

Bug Fixes

• [internal only] Build script improvement

9.3.0 (2024-05-17)

New Features

• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the GetEntity API.
• Added new Iterator property, "rocksdb.iterator.is-value-pinned", for checking whether the Slice returned by Iterator::value() can be used until the Iterator is destroyed.
• Optimistic transactions and WriteCommitted pessimistic transactions now support the MultiGetEntity API.
• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the PutEntity API. Support for read APIs and other write policies (WritePrepared, WriteUnprepared) will be added later.

Public API Changes

• Exposed block based metadata cache options via C API
• Exposed compaction pri via c api.
• Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.

Behavior Changes

• CompactRange() with change_level=true on a CF with FIFO compaction will return Status::NotSupported().
• External file ingestion with FIFO compaction will always ingest to L0.

Bug Fixes

• Fixed a bug for databases using DBOptions::allow_2pc == true (all TransactionDBs except OptimisticTransactionDB) that have exactly one column family. Due to a missing WAL sync, attempting to open the DB could have returned a Status::Corruption with a message like "SST file is ahead of WALs".
• Fix a bug in CreateColumnFamilyWithImport() where if multiple CFs are imported, we were not resetting files' epoch number and L0 files can have overlapping key range but the same epoch number.
• Fixed race conditions when ColumnFamilyOptions::inplace_update_support == true between user overwrites and reads on the same key.
• Fix a bug where CompactFiles() can compact files of range conflict with other ongoing compactions' when preclude_last_level_data_seconds > 0 is used
• Fixed a false positive Status::Corruption reported when reopening a DB that used DBOptions::recycle_log_file_num > 0 and DBOptions::wal_compression != kNoCompression.
• While WAL is locked with LockWAL(), some operations like Flush() and IngestExternalFile() are now blocked as they should have been.
• Fixed a bug causing stale memory access when using the TieredSecondaryCache with an NVM secondary cache, and a file system that supports return an FS allocated buffer for MultiRead (FSSupportedOps::kFSBuffer is set).

RocksDB 9.2.1

9.2.1 (2024-05-03)

Public API Changes

• Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.

9.2.0 (2024-05-01)

New Features

• Added two options deadline and max_size_bytes for CacheDumper to exit early
• Added a new API GetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for wide-column point lookups with read-your-own-writes consistency. Similarly to GetFromBatchAndDB, the API can combine data from the write batch with data from the underlying database if needed. See the API comments for more details.
• [Experimental] Introduce two new cross-column-family iterators - CoalescingIterator and AttributeGroupIterator. The CoalescingIterator enables users to iterate over multiple column families and access their values and columns. During this iteration, if the same key exists in more than one column family, the keys in the later column family will overshadow the previous ones. The AttributeGroupIterator allows users to gather wide columns per Column Family and create attribute groups while iterating over keys across all CFs.
• Added a new API MultiGetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for batched wide-column point lookups with read-your-own-writes consistency. Similarly to MultiGetFromBatchAndDB, the API can combine data from the write batch with data from the underlying database if needed. See the API comments for more details.
• *Adds a SstFileReader::NewTableIterator API to support programmatically read a SST file as a raw table file.
• Add an option to WaitForCompactOptions - wait_for_purge to make WaitForCompact() API wait for background purge to complete

Public API Changes

• DeleteRange() will return NotSupported() if row_cache is configured since they don't work together in some cases.
• Deprecated CompactionOptions::compression since CompactionOptions's API for configuring compression was incomplete, unsafe, and likely unnecessary
• Using OptionChangeMigration() to migrate from non-FIFO to FIFO compaction
  with Options::compaction_options_fifo.max_table_files_size > 0 can cause
  the whole DB to be dropped right after migration if the migrated data is larger than
  max_table_files_size

Behavior Changes

• Enabling BlockBasedTableOptions::block_align is now incompatible (i.e., APIs will return Status::InvalidArgument) with more ways of enabling compression: CompactionOptions::compression, ColumnFamilyOptions::compression_per_level, and ColumnFamilyOptions::bottommost_compression.
• Changed the default value of CompactionOptions::compression to kDisableCompressionOption, which means the compression type is determined by the ColumnFamilyOptions.
• BlockBasedTableOptions::optimize_filters_for_memory is now set to true by default. When partition_filters=false, this could lead to somewhat increased average RSS memory usage by the block cache, but this "extra" usage is within the allowed memory budget and should make memory usage more consistent (by minimizing internal fragmentation for more kinds of blocks).
• Dump all keys for cache dumper impl if SetDumpFilter() is not called
• CompactRange() with CompactRangeOptions::change_level = true and CompactRangeOptions::target_level = 0 that ends up moving more than 1 file from non-L0 to L0 will return Status::Aborted().
• On distributed file systems that support file system level checksum verification and reconstruction reads, RocksDB will now retry a file read if the initial read fails RocksDB block level or record level checksum verification. This applies to MANIFEST file reads when the DB is opened, and to SST file reads at all times.

Bug Fixes

• Fix a bug causing VerifyFileChecksums() to return false-positive corruption under BlockBasedTableOptions::block_align=true
• Provide consistent view of the database across the column families for NewIterators() API.
• Fixed feature interaction bug for DeleteRange() together with ColumnFamilyOptions::memtable_insert_with_hint_prefix_extractor. The impact of this bug would likely be corruption or crashing.
• Fixed hang in DisableManualCompactions() where compactions waiting to be scheduled due to conflicts would not be canceled promptly
• Fixed a regression when ColumnFamilyOptions::max_successive_merges > 0 where the CPU overhead for deciding whether to merge could have increased unless the user had set the option ColumnFamilyOptions::strict_max_successive_merges
• Fixed a bug in MultiGet() and MultiGetEntity() together with blob files (ColumnFamilyOptions::enable_blob_files == true). An error looking up one of the keys could cause the results to be wrong for other keys for which the statuses were Status::OK.
• Fixed a bug where wrong padded bytes are used to generate file checksum and DataVerificationInfo::checksum upon file creation
• Correctly implemented the move semantics of PinnableWideColumns.
• Fixed a bug when the recycle_log_file_num in DBOptions is changed from 0 to non-zero when a DB is reopened. On a subsequent reopen, if a log file created when recycle_log_file_num==0 was reused previously, is alive and is empty, we could end up inserting stale WAL records into the memtable.
• *Fix a bug where obsolete files' deletion during DB::Open are not rate limited with SstFilemManager's slow deletion feature even if it's configured.