References

SQL commands

Schema management
Data management
Backup
  • BACKUP - Backs up your tables
SELECT
Flushing misc things
  • FLUSH ATTRIBUTES - Forces flushing updated attributes to disk
  • FLUSH HOSTNAMES - Renews IPs associates to agent host names
  • FLUSH LOGS - Initiates reopen of searchd log and query log files (similar to USR1)
Real-time table optimization
Importing to a real-time table
  • ATTACH TABLE - Moves data from a plain table to a real-time table
  • IMPORT TABLE - Imports previously created RT or PQ table into a server running in the RT mode
Replication
Plain table rotate
Transactions
  • BEGIN - Begins a transaction
  • COMMIT - Finishes a transaction
  • ROLLBACK - Rolls back a transaction
CALL
Plugins
Server status
  • SHOW STATUS - Displays a number of useful performance counters
  • SHOW THREADS - Lists all currently active client threads
  • SHOW VARIABLES - Lists server-wide variables and their values
  • SHOW VERSION - Provides detailed version information of various components of the instance.

HTTP endpoints

  • /sql - Execute an SQL statement over HTTP JSON
  • /cli - Provides an HTTP command line interface
  • /insert - Inserts a document into a real-time table
  • /pq/tbl_name/doc - Adds a PQ rule to a percolate table
  • /update - Updates a document in a real-time table
  • /replace - Replaces an existing document in a real-time table or inserts it if it doesn't exist
  • /pq/tbl_name/doc/N?refresh=1 - Replaces a PQ rule in a percolate table
  • /delete - Removes a document from a table
  • /bulk - Executes multiple insert, update, or delete operations in a single call. Learn more about bulk inserts here.
  • /search - Performs a search
  • /search -> knn - Performs a KNN vector search
  • /pq/tbl_name/search - Performs a reverse search in a percolate table
  • /tbl_name/_mapping - Creates a table schema in the Elasticsearch style

Common things

Common table settings
Plain table settings
Distributed table settings
RT table settings

Full-text search operators

Functions

Mathematical
  • ABS() - Returns absolute value
  • ATAN2() - Returns arctangent function of two arguments
  • BITDOT() - Returns sum of products of each bit of a mask multiplied with its weight
  • CEIL() - Returns smallest integer value greater or equal to the argument
  • COS() - Returns cosine of the argument
  • CRC32() - Returns CRC32 value of the argument
  • EXP() - Returns exponent of the argument
  • FIBONACCI() - Returns the N-th Fibonacci number, where N is the integer argument
  • FLOOR() - Returns the largest integer value lesser or equal to the argument
  • GREATEST() - Takes JSON/MVA array as the argument and returns the greatest value in that array
  • IDIV() - Returns result of an integer division of the first argument by the second argument
  • LEAST() - Takes JSON/MVA array as the argument, and returns the least value in that array
  • LN() - Returns natural logarithm of the argument
  • LOG10() - Returns common logarithm of the argument
  • LOG2() - Returns binary logarithm of the argument
  • MAX() - Returns the larger of two arguments
  • MIN() - Returns the smaller of two arguments
  • POW() - Returns the first argument raised to the power of the second argument
  • RAND() - Returns random float between 0 and 1
  • SIN() - Returns sine of the argument
  • SQRT() - Returns square root of the argument
Searching and ranking
  • BM25F() - Returns precise BM25F formula value
  • EXIST() - Replaces non-existing columns with default values
  • GROUP_CONCAT() - Produces a comma-separated list of the attribute values of all documents in the group
  • HIGHLIGHT() - Highlights search results
  • MIN_TOP_SORTVAL() - Returns sort key value of the worst found element in the current top-N matches
  • MIN_TOP_WEIGHT() - Returns weight of the worst found element in the current top-N matches
  • PACKEDFACTORS() - Outputs weighting factors
  • REMOVE_REPEATS() - Removes repeated adjusted rows with the same 'column' value
  • WEIGHT() - Returns fulltext match score
  • ZONESPANLIST() - Returns pairs of matched zone spans
  • QUERY() - Returns current full-text query
Type casting
  • BIGINT() - Forcibly promotes the integer argument to 64-bit type
  • DOUBLE() - Forcibly promotes given argument to floating point type
  • INTEGER() - Forcibly promotes given argument to 64-bit signed type
  • TO_STRING() - Forcibly promotes the argument to string type
  • UINT() - Converts the given argument to 32-bit unsigned integer type
  • UINT64() - Converts the given argument to 64-bit unsigned integer type
  • SINT() - Interprets 32-bit unsigned integer as signed 64-bit integer
Arrays and conditions
  • ALL() - Returns 1 if condition is true for all elements in the array
  • ANY() - Returns 1 if condition is true for any element in the array
  • CONTAINS() - Checks whether the (x,y) point is within the given polygon
  • IF() - Checks whether the 1st argument is equal to 0.0, returns the 2nd argument if it is not zero or the 3rd one when it is
  • IN() - Returns 1 if the first argument is equal to any of the other arguments, or 0 otherwise
  • INDEXOF() - Iterates through all elements in the array and returns index of the first matching element
  • INTERVAL() - Returns index of the argument that is less than the first argument
  • LENGTH() - Returns number of elements in MVA
  • REMAP() - Allows to make some exceptions of expression values depending on the condition values
Date and time
  • NOW() - Returns current timestamp as an INTEGER
  • CURTIME() - Returns current time in local timezone
  • CURDATE() - Returns current date in local timezone
  • UTC_TIME() - Returns current time in UTC timezone
  • UTC_TIMESTAMP() - Returns current date/time in UTC timezone
  • SECOND() - Returns integer second from the timestamp argument
  • MINUTE() - Returns integer minute from the timestamp argument
  • HOUR() - Returns integer hour from the timestamp argument
  • DAY() - Returns integer day from the timestamp argument
  • MONTH() - Returns integer month from the timestamp argument
  • QUARTER() - Returns the integer quarter of the year from a timestamp argument
  • YEAR() - Returns integer year from the timestamp argument
  • DAYNAME() - Returns the weekday name for a given timestamp argument
  • MONTHNAME() - Returns the name of the month for a given timestamp argument
  • DAYOFWEEK() - Returns the integer weekday index for a given timestamp argument
  • DAYOFYEAR() - Returns the integer day of the year for a given timestamp argument
  • YEARWEEK() - Returns the integer year and the day code of the first day of current week for a given timestamp argument
  • YEARMONTH() - Returns integer year and month code from the timestamp argument
  • YEARMONTHDAY() - Returns integer year, month and day code from the timestamp argument
  • TIMEDIFF() - Returns difference between the timstamps
  • DATEDIFF() - Returns the number of days between two given timestamps
  • DATE() - Formats the date part from a timestamp argument
  • TIME() - Formats the time part from a timestamp argument
  • DATE_FORMAT() - Returns a formatted string based on the provided date and format arguments
Geo-spatial
  • GEODIST() - Computes geosphere distance between two given points
  • GEOPOLY2D() - Creates a polygon that takes in account the Earth's curvature
  • POLY2D() - Creates a simple polygon in plain space
String
  • CONCAT() - Concatenates two or more strings
  • REGEX() - Returns 1 if regular expression matched to string of attribute and 0 otherwise
  • SNIPPET() - Highlights search results
  • SUBSTRING_INDEX() - Returns a substring of the string before the specified number of delimiter occurs
  • Other
  • CONNECTION_ID() - Returns the current connection ID
  • KNN_DIST() - Returns KNN vector search distance
  • LAST_INSERT_ID() - Returns ids of documents inserted or replaced by last statement in the current session

Common settings in configuration file

To be put to section common {} in configuration file:

Indexer

indexer is a tool to create plain tables

Indexer settings in configuration file

To be put to section indexer {} in configuration file:

Indexer start parameters
indexer [OPTIONS] [indexname1 [indexname2 [...]]]
  • --all - Rebuilds all tables from the config
  • --buildstops - Analyzes the table source as if indexing the data, generating a list of indexed terms
  • --buildfreqs - Adds the frequency count to the table for --buildstops
  • --config, -c - Specifies the path to the configuration file
  • --dump-rows - Dumps rows retrieved by SQL source(s) into the specified file
  • --help - Displays all available parameters
  • --keep-attrs - Allows reuse of existing attributes when reindexing
  • --keep-attrs-names - Specifies which attributes to reuse from the existing table
  • --merge-dst-range - Applies the given filter range during merging
  • --merge-killlists - Alters kill list processing when merging tables
  • --merge - Combines two plain tables into one
  • --nohup - Prevents indexer from sending SIGHUP when this option is enabled
  • --noprogress - Hides progress details
  • --print-queries - Outputs SQL queries sent by the indexer to the database
  • --print-rt - Displays data fetched from SQL source(s) as INSERTs into a real-time table
  • --quiet - Suppresses all output
  • --rotate - Initiates table rotation after all tables are built
  • --sighup-each - Triggers rotation of each table after it's built
  • -v - Displays indexer version

Table Converter for Manticore v2 / Sphinx v2

index_converter is a tool designed to convert tables created with Sphinx/Manticore Search 2.x into the Manticore Search 3.x table format.

index_converter {--config /path/to/config|--path}
Table converter start parameters
  • --config, -c - Path to table configuration file
  • --index - Specifies which table to convert
  • --path - Sets path containing table(s) instead of the configuration file
  • --strip-path - Removes path from filenames referenced by table
  • --large-docid - Allows conversion of documents with ids larger than 2^63
  • --output-dir - Writes new files in a specified folder
  • --all - Converts all tables from the configuration file / path
  • --killlist-target - Sets target tables for applying kill-lists

Searchd

searchd is the Manticore server.

Searchd settings in a configuration file

To be put in the searchd {} section of the configuration file:

Searchd start parameters
searchd [OPTIONS]
  • --config, -c - Specifies the path to the configuration file
  • --console - Forces the server to run in console mode
  • --coredump - Enables core dump saving upon crash
  • --cpustats - Enables CPU time reporting
  • --delete - Removes the Manticore service from Microsoft Management Console and other locations where services are registered
  • --force-preread - Prevents the server from serving incoming connections until table files are pre-read
  • --help, -h - Displays all available parameters
  • --table (--index) - Restricts the server to serve only the specified table
  • --install - Installs searchd as a service in Microsoft Management Console
  • --iostats - Enables input/output reporting
  • --listen, -l - Overrides listen from the configuration file
  • --logdebug, --logdebugv, --logdebugvv - Enables additional debug output in the server log
  • --logreplication - Enables extra replication debug output in the server log
  • --new-cluster - Initializes a replication cluster and sets the server as a reference node with cluster restart protection
  • --new-cluster-force - Initializes a replication cluster and sets the server as a reference node, bypassing cluster restart protection
  • --nodetach - Keeps searchd running in the foreground
  • --ntservice - Used by Microsoft Management Console to launch searchd as a service on Windows platforms
  • --pidfile - Overrides pid_file in the configuration file
  • --port, p - Specifies the port searchd should listen on, ignoring the port specified in the configuration file
  • --replay-flags - Sets additional binary log replay options
  • --servicename - Assigns the given name to searchd when installing or deleting the service, as displayed in Microsoft Management Console
  • --status - Queries the running search service to return its status
  • --stop - Stops the Manticore server
  • --stopwait - Stops the Manticore server gracefully
  • --strip-path - Removes path names from all file names referenced in the table
  • -v - Displays version information
Searchd environment variables

Indextool

Assorted table maintenance features helpful for troubleshooting.

indextool <command> [options]
Indextool Start Parameters

Utilized for dumping various debug information related to the physical table.

indextool <command> [options]
  • --config, -c - Specifies the path to the configuration file
  • --quiet, -q - Keeps indextool quiet; no banner output, etc.
  • --help, -h - Lists all available parameters
  • -v - Displays version information
  • Indextool - Verifies the configuration file
  • --buildidf - Builds an IDF file from one or more dictionary dumps
  • --build-infixes - Builds infixes for an existing dict=keywords table
  • --dumpheader - Quickly dumps the provided table header file
  • --dumpconfig - Dumps table definition from the given table header file in a nearly compliant manticore.conf format
  • --dumpheader - Dumps table header by table name while looking up the header path in the configuration file
  • --dumpdict - Dumps the table dictionary
  • --dumpdocids - Dumps document IDs by table name
  • --dumphitlist - Dumps all occurrences of the given keyword/id in the specified table
  • --docextract - Runs table check pass on entire dictionary/docs/hits and collects all words and hits belonging to the requested document
  • --fold - Tests tokenization based on table settings
  • --htmlstrip - Filters STDIN using HTML stripper settings for the specified table
  • --mergeidf - Merges multiple .idf files into a single file
  • --morph - Applies morphology to the provided STDIN and outputs the result to stdout
  • --check - Checks table data files for consistency
  • --check-id-dups - Checks for duplicate IDs
  • --check-disk-chunk - Checks a single disk chunk of an RT table
  • --strip-path - Removes path names from all file names referenced in the table
  • --rotate - Determines whether to check a table waiting for rotation when using --check
  • --apply-killlists - Applies kill-lists for all tables listed in the configuration file

Wordbreaker

Splits compound words into their components.

wordbreaker [-dict path/to/dictionary_file] {split|test|bench}
Wordbreaker start parameters
  • STDIN - Accepts a string to break into parts
  • -dict - Specifies the dictionary file to use
  • split|test|bench - Specifies the command

Spelldump

Extracts the contents of a dictionary file using ispell or MySpell format

spelldump [options] <dictionary> <affix> [result] [locale-name]
  • dictionary - Main dictionary file
  • affix - Affix file for the dictionary
  • result - Specifies the output destination for the dictionary data
  • locale-name - Specifies the locale details to use

List of reserved keywords

A comprehensive alphabetical list of keywords currently reserved in Manticore SQL syntax (thus, they cannot be used as identifiers).

AND, AS, BY, COLUMNARSCAN, DATE_ADD, DATE_SUB, DAY, DISTINCT, DIV, DOCIDINDEX, EXPLAIN, FACET, FALSE, FORCE, FROM, HOUR, IGNORE, IN, INTERVAL, INDEXES, INNER, IS, JOIN, KNN, LEFT, LIMIT, MINUTE, MOD, MONTH, NOT, NO_COLUMNARSCAN, NO_DOCIDINDEX, NO_SECONDARYINDEX, NULL, OFFSET, ON, OR, ORDER, QUARTER, REGEX, RELOAD, SECOND, SECONDARYINDEX, SELECT, SYSFILTERS, TRUE, WEEK, YEAR

Documentation for old Manticore versions

Reporting bugs