Over time, RT tables may become fragmented into numerous disk chunks and/or contaminated with deleted, yet unpurged data, affecting search performance. In these cases, optimization is necessary. Essentially, the optimization process combines pairs of disk chunks, removing documents that were previously deleted using DELETE statements.
Beginning with Manticore 4, this process occurs automatically by default. However, you can also use the following commands to manually initiate table compaction.
OPTIMIZE TABLE index_name [OPTION opt_name = opt_value [,...]]
OPTIMIZE statement adds an RT table to the optimization queue, which will be processed in a background thread.
OPTIMIZE TABLE rt;
By default, OPTIMIZE merges the RT table's disk chunks down to a number equal to
# of CPU cores * 2. You can control the number of optimized disk chunks using the
Additional options include:
OPTIMIZE TABLE rt OPTION cutoff=4;
OPTION sync=1 (0 by default), the command will wait for the optimization process to complete before returning. If the connection is interrupted, the optimization will continue running on the server.
OPTIMIZE TABLE rt OPTION sync=1;
Optimization can be a lengthy and I/O-intensive process. To minimize the impact, all actual merge work is executed serially in a special background thread, and the
OPTIMIZE statement simply adds a job to its queue. The optimization thread can be I/O-throttled, and you can control the maximum number of I/Os per second and the maximum I/O size with the rt_merge_iops and rt_merge_maxiosize directives, respectively.
During optimization, the RT table being optimized remains online and available for both searching and updates nearly all the time. It is locked for a very brief period when a pair of disk chunks is successfully merged, allowing for the renaming of old and new files and updating the table header.
As long as auto_optimize is not disabled, tables are optimized automatically.
If you are experiencing unexpected SSTs or want tables across all nodes of the cluster to be binary identical, you need to:
- Disable auto_optimize.
- Manually optimize tables:
On one of the nodes, drop the table from the cluster:‹›
ALTER CLUSTER mycluster DROP myindex;
Optimize the table:‹›
OPTIMIZE TABLE myindex;
Add back the table to the cluster:‹›
ALTER CLUSTER mycluster ADD myindex;
When the table is added back, the new files created by the optimization process will be replicated to the other nodes in the cluster. Any local changes made to the table on other nodes will be lost.
Table data modifications (inserts, replaces, deletes, updates) should either:
- Be postponed, or
- Be directed to the node where the optimization process is running.
Note that while the table is out of the cluster, insert/replace/delete/update commands should refer to it without the cluster name prefix (for SQL statements or the cluster property in case of an HTTP JSON request), otherwise they will fail. Once the table is added back to the cluster, you must resume write operations on the table and include the cluster name prefix again, or they will fail.
Search operations are available as usual during the process on any of the nodes.
Manticore provides isolation during the flushing and merging process of a real-time table to prevent any changes from affecting running queries.
For example, during table compaction, a pair of disk chunks are merged and a new chunk is produced. At one point, a new version of the table is created with the new chunk replacing the original pair. This is done seamlessly so that a long-running query using the original chunks will continue to see the old version of the table, while a new query will see the new version with the resulting merged chunk.
The same applies to flushing a RAM chunk, where suitable RAM segments are merged into a new disk chunk and the participated RAM chunk segments are abandoned. During this operation, Manticore provides isolation for queries that started before the operation began.
Furthermore, these operations are transparent for replaces and updates. If you update an attribute in a document that belongs to a disk chunk being merged with another one, the update will be applied to both that chunk and the resulting merged chunk. If you delete a document during a merge, it will be deleted in the original chunk and also in the resulting merged chunk, which will either have the document marked as deleted or have no such document at all if the deletion happened early in the merging process.
FREEZE tbl1[, tbl2, ...]
FREEZE readies a real-time/plain table for a secure backup. Specifically, it:
- Deactivates table compaction. If the table is currently being compacted,
FREEZEwill wait for completion.
- Transfers the current RAM chunk to a disk chunk.
- Flushes attributes.
- Disables implicit operations that could modify the disk files.
- Shows the actual file list associated with the table.
The built-in tool manticore-backup uses
FREEZE to ensure data consistency. You can do the same if you want to create your own backup solution or need to freeze tables for other reasons. Just follow these steps:
FREEZEa table (or a few).
- Capture the output of the
FREEZEcommand and back up the specified files.
UNFREEZEthe table(s) once finished.
+-------------------+---------------------------------+ | file | normalized | +-------------------+---------------------------------+ | data/t/t.0.spa | /work/anytest/data/t/t.0.spa | | data/t/t.0.spd | /work/anytest/data/t/t.0.spd | | data/t/t.0.spds | /work/anytest/data/t/t.0.spds | | data/t/t.0.spe | /work/anytest/data/t/t.0.spe | | data/t/t.0.sph | /work/anytest/data/t/t.0.sph | | data/t/t.0.sphi | /work/anytest/data/t/t.0.sphi | | data/t/t.0.spi | /work/anytest/data/t/t.0.spi | | data/t/t.0.spm | /work/anytest/data/t/t.0.spm | | data/t/t.0.spp | /work/anytest/data/t/t.0.spp | | data/t/t.0.spt | /work/anytest/data/t/t.0.spt | | data/t/t.meta | /work/anytest/data/t/t.meta | | data/t/t.ram | /work/anytest/data/t/t.ram | | data/t/t.settings | /work/anytest/data/t/t.settings | +-------------------+---------------------------------+ 13 rows in set (0.01 sec)
file column indicates the table's file paths within the data_dir of the running instance. The
normalized column displays the absolute paths for the same files. To back up a table, simply copy the provided files without additional preparation.
When a table is frozen, you cannot execute
UPDATE queries; they will fail with the error message "index is locked now, try again later."
REPLACE queries have some restrictions while the table is frozen:
DELETEaffects a document in the current RAM chunk - it is permitted.
DELETEimpacts a document in a disk chunk but was previously deleted - it is allowed.
DELETEwould alter an actual disk chunk - it will wait until the table is unfrozen.
FLUSHing a RAM chunk of a frozen table will report 'success', but no real saving will occur.
TRUNCATE of a frozen table is allowed since these operations are not implicit. We assume that if you truncate or drop a table, you don't need it backed up; therefore, it should not have been frozen initially.
INSERTing into a frozen table is supported but limited: new data will be stored in RAM (as usual) until
rt_mem_limit is reached; then, new insertions will wait until the table is unfrozen.
If you shut down the daemon with a frozen table, it will act as if it experienced a dirty shutdown (e.g.,
kill -9): newly inserted data will not be saved in the RAM-chunk on disk, and upon restart, it will be restored from a binary log (if any) or lost (if binary logging is disabled).
UNFREEZE tbl1[, tbl2, ...]
UNFREEZE reactivates previously blocked operations and resumes the internal compaction service. All operations waiting for a table to unfreeze will also be unfrozen and complete normally.