Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
P
Python Rocksdb
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Wiki
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
public
Python Rocksdb
Commits
7e1df8be
Commit
7e1df8be
authored
10 years ago
by
hofmockel
Browse files
Options
Downloads
Patches
Plain Diff
Document the changes done at the options.
parent
87a7ddfe
No related branches found
Branches containing commit
No related tags found
No related merge requests found
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
docs/api/options.rst
+52
-97
52 additions, 97 deletions
docs/api/options.rst
docs/changelog.rst
+13
-1
13 additions, 1 deletion
docs/changelog.rst
docs/tutorial/index.rst
+5
-3
5 additions, 3 deletions
docs/tutorial/index.rst
with
70 additions
and
101 deletions
docs/api/options.rst
+
52
−
97
View file @
7e1df8be
...
@@ -109,45 +109,6 @@ Options object
...
@@ -109,45 +109,6 @@ Options object
| *Type:* ``int``
| *Type:* ``int``
| *Default:* ``5000``
| *Default:* ``5000``
.. py:attribute:: block_cache
Control over blocks (user data is stored in a set of blocks, and
a block is the unit of reading from disk).
If not ``None`` use the specified cache for blocks.
If ``None``, rocksdb will automatically create and use an 8MB internal cache.
| *Type:* Instace of :py:class:`rocksdb.LRUCache`
| *Default:* ``None``
.. py:attribute:: block_cache_compressed
If not ``None`` use the specified cache for compressed blocks.
If ``None``, rocksdb will not use a compressed block cache.
| *Type:* Instace of :py:class:`rocksdb.LRUCache`
| *Default:* ``None``
.. py:attribute:: block_size
Approximate size of user data packed per block. Note that the
block size specified here corresponds to uncompressed data. The
actual size of the unit read from disk may be smaller if
compression is enabled. This parameter can be changed dynamically.
| *Type:* ``int``
| *Default:* ``4096``
.. py:attribute:: block_restart_interval
Number of keys between restart points for delta encoding of keys.
This parameter can be changed dynamically. Most clients should
leave this parameter alone.
| *Type:* ``int``
| *Default:* ``16``
.. py:attribute:: compression
.. py:attribute:: compression
Compress blocks using the specified compression algorithm.
Compress blocks using the specified compression algorithm.
...
@@ -156,15 +117,6 @@ Options object
...
@@ -156,15 +117,6 @@ Options object
| *Type:* Member of :py:class:`rocksdb.CompressionType`
| *Type:* Member of :py:class:`rocksdb.CompressionType`
| *Default:* :py:attr:`rocksdb.CompressionType.snappy_compression`
| *Default:* :py:attr:`rocksdb.CompressionType.snappy_compression`
.. py:attribute:: whole_key_filtering
If ``True``, place whole keys in the filter (not just prefixes).
This must generally be true for gets to be efficient.
| *Type:* ``bool``
| *Default:* ``True``
.. py:attribute:: num_levels
.. py:attribute:: num_levels
Number of levels for this database
Number of levels for this database
...
@@ -315,15 +267,6 @@ Options object
...
@@ -315,15 +267,6 @@ Options object
| *Type:* ``bool``
| *Type:* ``bool``
| *Default:* ``False``
| *Default:* ``False``
.. py:attribute:: db_stats_log_interval
This number controls how often a new scribe log about
db deploy stats is written out.
-1 indicates no logging at all.
| *Type:* ``int``
| *Default:* ``1800``
.. py:attribute:: db_log_dir
.. py:attribute:: db_log_dir
This specifies the info LOG dir.
This specifies the info LOG dir.
...
@@ -346,16 +289,6 @@ Options object
...
@@ -346,16 +289,6 @@ Options object
| *Type:* ``unicode``
| *Type:* ``unicode``
| *Default:* ``""``
| *Default:* ``""``
.. py:attribute:: disable_seek_compaction
Disable compaction triggered by seek.
With bloomfilter and fast storage, a miss on one level
is very cheap if the file handle is cached in table cache
(which is true if max_open_files is large).
| *Type:* ``bool``
| *Default:* ``True``
.. py:attribute:: delete_obsolete_files_period_micros
.. py:attribute:: delete_obsolete_files_period_micros
The periodicity when obsolete files get deleted. The default
The periodicity when obsolete files get deleted. The default
...
@@ -454,15 +387,6 @@ Options object
...
@@ -454,15 +387,6 @@ Options object
| *Type:* ``int``
| *Type:* ``int``
| *Default:* ``(2**64) - 1``
| *Default:* ``(2**64) - 1``
.. py:attribute:: no_block_cache
Disable block cache. If this is set to true,
then no block cache should be used, and the block_cache should
point to ``None``
| *Type:* ``bool``
| *Default:* ``False``
.. py:attribute:: table_cache_numshardbits
.. py:attribute:: table_cache_numshardbits
Number of shards used for table cache.
Number of shards used for table cache.
...
@@ -579,17 +503,6 @@ Options object
...
@@ -579,17 +503,6 @@ Options object
| *Type:* ``int``
| *Type:* ``int``
| *Default:* ``3600``
| *Default:* ``3600``
.. py:attribute:: block_size_deviation
This is used to close a block before it reaches the configured
'block_size'. If the percentage of free space in the current block is less
than this specified number and adding a new record to the block will
exceed the configured block size, then this block will be closed and the
new record will be written to the next block.
| *Type:* ``int``
| *Default:* ``10``
.. py:attribute:: advise_random_on_open
.. py:attribute:: advise_random_on_open
If set true, will hint the underlying file system that the file
If set true, will hint the underlying file system that the file
...
@@ -797,15 +710,6 @@ Options object
...
@@ -797,15 +710,6 @@ Options object
*Default:* ``None``
*Default:* ``None``
.. py:attribute:: filter_policy
If not ``None`` use the specified filter policy to reduce disk reads.
A python filter policy must implement the
:py:class:`rocksdb.interfaces.FilterPolicy` interface.
Recommendes is a instance of :py:class:`rocksdb.BloomFilterPolicy`
*Default:* ``None``
.. py:attribute:: prefix_extractor
.. py:attribute:: prefix_extractor
If not ``None``, use the specified function to determine the
If not ``None``, use the specified function to determine the
...
@@ -902,7 +806,8 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
...
@@ -902,7 +806,8 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
Wraps BlockBasedTableFactory of RocksDB.
Wraps BlockBasedTableFactory of RocksDB.
.. py:method:: __init__(index_type='binary_search', hash_index_allow_collision=True, checksum='crc32')
.. py:method:: __init__(index_type='binary_search', hash_index_allow_collision=True, checksum='crc32', block_cache, block_cache_compressed, filter_policy=None, no_block_cache=False, block_size=None, block_size_deviation=None, block_restart_interval=None, whole_key_filtering=None):
:param string index_type:
:param string index_type:
* ``binary_search`` a space efficient index block that is optimized
* ``binary_search`` a space efficient index block that is optimized
...
@@ -922,6 +827,56 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
...
@@ -922,6 +827,56 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
even though they have different checksum type.
even though they have different checksum type.
Can be either ``crc32`` or ``xxhash``.
Can be either ``crc32`` or ``xxhash``.
:param block_cache:
Control over blocks (user data is stored in a set of blocks, and
a block is the unit of reading from disk).
If ``None``, rocksdb will automatically create and use an 8MB internal cache.
If not ``None`` use the specified cache for blocks. In that case it must
be an instance of :py:class:`rocksdb.LRUCache`
:param block_cache_compressed:
If ``None``, rocksdb will not use a compressed block cache.
If not ``None`` use the specified cache for compressed blocks. In that
case it must be an instance of :py:class:`rocksdb.LRUCache`
:param filter_policy:
If not ``None`` use the specified filter policy to reduce disk reads.
A python filter policy must implement the
:py:class:`rocksdb.interfaces.FilterPolicy` interface.
Recommended is a instance of :py:class:`rocksdb.BloomFilterPolicy`
:param bool no_block_cache:
Disable block cache. If this is set to true,
then no block cache should be used, and the block_cache should
point to ``None``
:param int block_size:
If set to ``None`` the rocksdb default of ``4096`` is used.
Approximate size of user data packed per block. Note that the
block size specified here corresponds to uncompressed data. The
actual size of the unit read from disk may be smaller if
compression is enabled. This parameter can be changed dynamically.
:param int block_size_deviation:
If set to ``None`` the rocksdb default of ``10`` is used.
This is used to close a block before it reaches the configured
'block_size'. If the percentage of free space in the current block is less
than this specified number and adding a new record to the block will
exceed the configured block size, then this block will be closed and the
new record will be written to the next block.
:param int block_restart_interval:
If set to ``None`` the rocksdb default of ``16`` is used.
Number of keys between restart points for delta encoding of keys.
This parameter can be changed dynamically. Most clients should
leave this parameter alone.
:param bool whole_key_filtering:
If set to ``None`` the rocksdb default of ``True`` is used.
If ``True``, place whole keys in the filter (not just prefixes).
This must generally be true for gets to be efficient.
.. py:class:: rocksdb.PlainTableFactory
.. py:class:: rocksdb.PlainTableFactory
Plain Table with prefix-only seek. It wraps rocksdb PlainTableFactory.
Plain Table with prefix-only seek. It wraps rocksdb PlainTableFactory.
...
...
This diff is collapsed.
Click to expand it.
docs/changelog.rst
+
13
−
1
View file @
7e1df8be
...
@@ -35,7 +35,20 @@ For DB.iterkeys and DB.iteritems ``itertools.takewhile`` is a possible solution.
...
@@ -35,7 +35,20 @@ For DB.iterkeys and DB.iteritems ``itertools.takewhile`` is a possible solution.
**SST Table Builders:**
**SST Table Builders:**
* Removed ``NewTotalOrderPlainTableFactory``, because rocksdb drops it too.
* Removed ``NewTotalOrderPlainTableFactory``, because rocksdb drops it too.
**Changed Options:**
In newer versions of rocksdb a bunch of options were moved or removed.
* Rename ``bloom_bits_per_prefix`` of :py:class:`rocksdb.PlainTableFactory` to ``bloom_bits_per_key``
* Rename ``bloom_bits_per_prefix`` of :py:class:`rocksdb.PlainTableFactory` to ``bloom_bits_per_key``
* Removed ``Options.db_stats_log_interval``.
* Removed ``Options.disable_seek_compaction``
* Moved ``Options.no_block_cache`` to ``BlockBasedTableFactory``
* Moved ``Options.block_size`` to ``BlockBasedTableFactory``
* Moved ``Options.block_size_deviation`` to ``BlockBasedTableFactory``
* Moved ``Options.block_restart_interval`` to ``BlockBasedTableFactory``
* Moved ``Options.whole_key_filtering`` to ``BlockBasedTableFactory``
New:
New:
^^^^
^^^^
...
@@ -44,7 +57,6 @@ New:
...
@@ -44,7 +57,6 @@ New:
* Add more option to :py:class:`rocksdb.PlainTableFactory`
* Add more option to :py:class:`rocksdb.PlainTableFactory`
Version 0.2
Version 0.2
-----------
-----------
...
...
This diff is collapsed.
Click to expand it.
docs/tutorial/index.rst
+
5
−
3
View file @
7e1df8be
...
@@ -20,9 +20,11 @@ A more production ready open can look like this ::
...
@@ -20,9 +20,11 @@ A more production ready open can look like this ::
opts.write_buffer_size = 67108864
opts.write_buffer_size = 67108864
opts.max_write_buffer_number = 3
opts.max_write_buffer_number = 3
opts.target_file_size_base = 67108864
opts.target_file_size_base = 67108864
opts.filter_policy = rocksdb.BloomFilterPolicy(10)
opts.block_cache = rocksdb.LRUCache(2 * (1024 ** 3))
opts.table_factory = rocksdb.BlockBasedTableFactory(
opts.block_cache_compressed = rocksdb.LRUCache(500 * (1024 ** 2))
filter_policy=rocksdb.BloomFilterPolicy(10),
block_cache=rocksdb.LRUCache(2 * (1024 ** 3)),
block_cache_compressed=rocksdb.LRUCache(500 * (1024 ** 2)))
db = rocksdb.DB("test.db", opts)
db = rocksdb.DB("test.db", opts)
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment