You can also start Manticore Search by calling searchd
(Manticore Search server binary) directly:
searchd [OPTIONS]
Note that without specifying a path to the configuration file, searchd
will try to find it in several locations depending on the operating system.
The options available to searchd
in all operating systems are:
-
--help
(-h
for short) lists all of the parameters that can be used in your particular build ofsearchd
. -
--version
(-v
for short) shows Manticore Search version information. -
--config <file>
(-c <file>
for short) tellssearchd
to use the specified file as its configuration. -
--stop
is used to asynchronously stopsearchd
, using the details of the PID file as specified in the Manticore configuration file. Therefore, you may also need to confirm tosearchd
which configuration file to use with the--config
option. Example:$ searchd --config /etc/manticoresearch/manticore.conf --stop
-
--stopwait
is used to synchronously stopsearchd
.--stop
essentially tells the running instance to exit (by sending it a SIGTERM) and then immediately returns.--stopwait
will also attempt to wait until the runningsearchd
instance actually finishes the shutdown (eg. saves all the pending attribute changes) and exits. Example:$ searchd --config /etc/manticoresearch/manticore.conf --stopwait
Possible exit codes are as follows:
-
0 on success
-
1 if connection to running searchd server failed
-
2 if server reported an error during shutdown
-
3 if server crashed during shutdown
-
--status
command is used to query runningsearchd
instance status using the connection details from the (optionally) provided configuration file. It will try to connect to running instance using the first found UNIX socket or TCP port from the configuration file. On success it will query for a number of status and performance counter values and print them. You can also use SHOW STATUS command to access the very same counters via SQL protocol. Examples:$ searchd --status $ searchd --config /etc/manticoresearch/manticore.conf --status
-
--pidfile
is used to explicitly force using a PID file (where thesearchd
process identification number is stored) despite any other debugging options that say otherwise (for instance,--console
). This is a debugging option.$ searchd --console --pidfile
-
--console
is used to forcesearchd
into console mode. Typically, Manticore runs as a conventional server application and logs information into log files (as specified in the configuration file). However, when debugging issues in the configuration or the server itself, or trying to diagnose hard-to-track-down problems, it may be easier to force it to dump information directly to the console/command line from which it is being called. Running in console mode also means that the process will not be forked (so searches are done in sequence) and logs will not be written to. (It should be noted that console mode is not the intended method for runningsearchd
.) You can invoke it as:$ searchd --config /etc/manticoresearch/manticore.conf --console
-
--logdebug
,--logreplication
,--logdebugv
, and--logdebugvv
options enable additional debug output in the server log. They differ by the logging verboseness level. These are debugging options and should not be normally enabled, as they can pollute the log a lot. They can be used temporarily on request to assist with complicated debugging sessions. -
--iostats
is used in conjunction with the logging options (thequery_log
must have been activated inmanticore.conf
) to provide more detailed information on a per-query basis about the input/output operations carried out in the course of that query, with a slight performance hit and slightly bigger logs. The IO statistics don't include information about IO operations for attributes, as these are loaded with mmap. To enable it, you can startsearchd
as follows:$ searchd --config /etc/manticoresearch/manticore.conf --iostats
-
--cpustats
is used to provide actual CPU time report (in addition to wall time) in both query log file (for every given query) and status report (aggregated). It depends onclock_gettime()
Linux system call or falls back to less precise call on certain systems. You might startsearchd
thus:$ searchd --config /etc/manticoresearch/manticore.conf --cpustats
-
--port portnumber
(-p
for short) is used to specify the port that Manticore should listen on to accept binary protocol requests, usually for debugging purposes. This will usually default to 9312, but sometimes you need to run it on a different port. Specifying it on the command line will override anything specified in the configuration file. The valid range is 0 to 65535, but ports numbered 1024 and below usually require a privileged account in order to run.An example of usage:
$ searchd --port 9313
-
--listen ( address ":" port | port | path ) [ ":" protocol ]
(or-l
for short) Works as--port
, but allows you to specify not only the port, but the full path, IP address and port, or Unix-domain socket path thatsearchd
will listen on. In other words, you can specify either an IP address (or hostname) and port number, just a port number, or a Unix socket path. If you specify a port number but not the address, searchd will listen on all network interfaces. A Unix path is identified by a leading slash. As the last parameter, you can also specify a protocol handler (listener) to be used for connections on this socket. Supported protocol values are 'sphinx' and 'mysql' (MySQL protocol used since 4.1). -
--force-preread
forbids the server from serving any incoming connection until prereading of table files completes. By default, at startup, the server accepts connections while table files are lazy-loaded into memory. This extends the behavior and makes it wait until the files are loaded. -
--index (--table) <table>
(or-i (-t) <table>
for short) forces this instance ofsearchd
to only serve the specified table. Like--port
, above, this is usually for debugging purposes; more long-term changes would generally be applied to the configuration file itself. -
--strip-path
strips the path names from all the file names referenced from the table (stopwords, wordforms, exceptions, etc). This is useful for picking up tables built on another machine with possibly different path layouts. -
--replay-flags=<OPTIONS>
switch can be used to specify a list of extra binary log replay options. The supported options are:accept-desc-timestamp
, ignore descending transaction timestamps and replay such transactions anyway (the default behavior is to exit with an error).ignore-open-errors
, ignore missing binlog files (the default behavior is to exit with an error).ignore-trx-errors
, ignore any transaction errors and skip current binlog file (the default behavior is to exit with an error).ignore-all-errors
, ignore any errors described above (the default behavior is to exit with an error).
Example:
$ searchd --replay-flags=accept-desc-timestamp
-
--coredump
is used to enable saving a core file or a minidump of the server on crash. Disabled by default to speed up of server restart on crash. This is useful for debugging purposes.$ searchd --config /etc/manticoresearch/manticore.conf --coredump
-
--new-cluster
bootstraps a replication cluster and makes the server a reference node with cluster restart protection. On Linux you can also runmanticore_new_cluster
. It will start Manticore in--new-cluster
mode via systemd. -
--new-cluster-force
bootstraps a replication cluster and makes the server a reference node bypassing cluster restart protection. On Linux you can also runmanticore_new_cluster --force
. It will start Manticore in--new-cluster-force
mode via systemd.
There are some options for searchd
that are specific to Windows platforms, concerning handling as a service, and are only available in Windows binaries.
Note that in Windows searchd will default to --console
mode, unless you install it as a service.
-
--install
installssearchd
as a service into the Microsoft Management Console (Control Panel / Administrative Tools / Services). Any other parameters specified on the command line, where--install
is specified will also become part of the command line on future starts of the service. For example, as a part of callingsearchd
, you will likely also need to specify the configuration file with--config
, and you would do that as well as specifying--install
. Once called, the usual start/stop facilities will become available via the management console, so any methods you could use for starting, stopping and restarting services would also apply tosearchd
. Example:C:\WINDOWS\system32> C:\Manticore\bin\searchd.exe --install --config C:\Manticore\manticore.conf
If you want to have the I/O stats every time you start
searchd
, you need to specify the option on the same line as the--install
command thus:C:\WINDOWS\system32> C:\Manticore\bin\searchd.exe --install --config C:\Manticore\manticore.conf --iostats
-
--delete
removes the service from the Microsoft Management Console and other places where services are registered, after previously being installed with--install
. Note that this does not uninstall the software or delete the tables. It means the service will not be called from the services system, and will not be started on the machine's next start. If currently running as a service, the current instance will not be terminated (until the next reboot or until--stop
). If the service was installed with a custom name (with--servicename
), the same name will need to be specified with--servicename
when calling to uninstall. Example:C:\WINDOWS\system32> C:\Manticore\bin\searchd.exe --delete
-
--servicename <name>
applies the given name tosearchd
when installing or deleting the service, as it would appear in the Management Console; this will default to searchd, but if being deployed on servers where multiple administrators may log in to the system, or a system with multiplesearchd
instances, a more descriptive name may be applicable. Note that unless combined with--install
or--delete
, this option does not do anything. Example:C:\WINDOWS\system32> C:\Manticore\bin\searchd.exe --install --config C:\Manticore\manticore.conf --servicename ManticoreSearch
-
--ntservice
is an option that is passed by the Microsoft Management Console tosearchd
to invoke it as a service on Windows platforms. It would not normally be necessary to call this directly; this would normally be called by Windows when the service is started, although if you wanted to call this as a regular service from the command-line (as the complement to--console
) you could do so in theory. -
--safetrace
forcessearchd
to only use the system's backtrace() call in crash reports. In certain (rare) scenarios, this might be a "safer" way to get that report. This is a debugging option. -
--nodetach
switch (Linux only) tellssearchd
not to detach into the background. This will also cause log entries to be printed out to the console. Query processing operates as usual. This is a debugging option and might also be useful when you run Manticore in a Docker container to capture its output.
Manticore utilizes the plugin_dir for storing and using Manticore Buddy plugins. By default, this value is accessible to the "manticore" user in a standard installation. However, if you start the searchd daemon manually with a different user, the daemon might not have access to the plugin_dir
. To address this problem, ensure you specify a plugin_dir
in the common section that the user running the searchd daemon can write to.
searchd
supports a number of signals:
SIGTERM
- Initiates a clean shutdown. New queries will not be handled, but queries that are already started will not be forcibly interrupted.SIGHUP
- Initiates tables rotation. Depending on the value of seamless_rotate setting, new queries might be shortly stalled; clients will receive temporary errors.SIGUSR1
- Forces reopen of searchd log and query log files, allowing for log file rotation.
MANTICORE_TRACK_DAEMON_SHUTDOWN=1
enables detailed logging while searchd is shutting down. It's useful in case of some shutdown problems, such as when Manticore takes too long to shut down or freezes during the shutdown process.
The image is based on current release of Manticore package.
The default configuration includes a sample Real-Time table and listens on the default ports:
9306
for connections from a MySQL client9308
for connections via HTTP9312
for connections via a binary protocol (e.g. in case you run a cluster)
The image comes with libraries for easy indexing data from MySQL, PostgreSQL, XML and CSV files.
The below is the simplest way to start Manticore in a container and log in to it via the mysql client:
docker run -e EXTRA=1 --name manticore --rm -d manticoresearch/manticore && echo "Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time" && until docker logs manticore 2>&1 | grep -q "accepting connections"; do sleep 1; echo -n .; done && echo && docker exec -it manticore mysql && docker stop manticore
Note that upon exiting the MySQL client, the Manticore container will be stopped and removed, resulting in no saved data. For information on using Manticore in a production environment, please see below.
The image comes with a sample table that can be loaded like this:
mysql> source /sandbox.sql
Also, the mysql client has several sample queries in its history that you can run on the above table, just use Up/Down keys in the client to see and run them.
For data persistence the folder /var/lib/manticore/
should be mounted to local storage or other desired storage engine.
The configuration file inside the instance is located at /etc/manticoresearch/manticore.conf
. For custom settings, this file should be mounted to your own configuration file.
The ports are 9306/9308/9312 for SQL/HTTP/Binary, expose them depending on how you are going to use Manticore. For example:
docker run -e EXTRA=1 --name manticore -v $(pwd)/data:/var/lib/manticore -p 127.0.0.1:9306:9306 -p 127.0.0.1:9308:9308 -d manticoresearch/manticore
or
docker run -e EXTRA=1 --name manticore -v $(pwd)/manticore.conf:/etc/manticoresearch/manticore.conf -v $(pwd)/data:/var/lib/manticore/ -p 127.0.0.1:9306:9306 -p 127.0.0.1:9308:9308 -d manticoresearch/manticore
Make sure to remove 127.0.0.1:
if you want the ports to be available for external hosts.
The Manticore Search Docker image doesn't come with the Manticore Columnar Library pre-installed, which is necessary if you require columnar storage and secondary indexes. However, it can easily be enabled during runtime by setting the environment variable EXTRA=1
. For example, docker run -e EXTRA=1 ... manticoresearch/manticore
. This will download and install the library in the data directory (which is typically mapped as a volume in production environments) and it won't be re-downloaded unless the Manticore Search version is changed.
Using EXTRA=1
also activates Manticore Buddy, which is used for processing certain commands. For more information, refer to the changelog.
If you only need the MCL, you can use the environment variable MCL=1
.
In many cases, you may want to use Manticore in conjunction with other images specified in a Docker Compose YAML file. Below is the minimal recommended configuration for Manticore Search in a docker-compose.yml file:
version: '2.2'
services:
manticore:
container_name: manticore
image: manticoresearch/manticore
environment:
- EXTRA=1
restart: always
ports:
- 127.0.0.1:9306:9306
- 127.0.0.1:9308:9308
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
memlock:
soft: -1
hard: -1
volumes:
- ./data:/var/lib/manticore
# - ./manticore.conf:/etc/manticoresearch/manticore.conf # uncomment if you use a custom config
Besides using the exposed ports 9306 and 9308, you can log into the instance by running docker-compose exec manticore mysql
.
HTTP protocol is exposed on port 9308. You can map the port locally and connect using curl.:
docker run -e EXTRA=1 --name manticore -p 9308:9308 -d manticoresearch/manticore
Create a table:
- JSON
POST /cli -d 'CREATE TABLE testrt ( title text, content text, gid integer)'
Insert a document:
- JSON
POST /insert
-d'{"table":"testrt","id":1,"doc":{"title":"Hello","content":"world","gid":1}}'
Perform a simple search:
- JSON
POST /search -d '{"table":"testrt","query":{"match":{"*":"hello world"}}}'
By default, the server is set to send its logging to /dev/stdout
, which can be viewed from the host with:
docker logs manticore
The query log can be diverted to Docker log by passing the variable QUERY_LOG_TO_STDOUT=true
.
Here is a simple docker-compose.yml
for defining a two node cluster:
version: '2.2'
services:
manticore-1:
image: manticoresearch/manticore
environment:
- EXTRA=1
restart: always
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
memlock:
soft: -1
hard: -1
networks:
- manticore
manticore-2:
image: manticoresearch/manticore
environment:
- EXTRA=1
restart: always
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
memlock:
soft: -1
hard: -1
networks:
- manticore
networks:
manticore:
driver: bridge
-
Start it:
docker-compose up
-
Create a cluster:
$ docker-compose exec manticore-1 mysql mysql> CREATE TABLE testrt ( title text, content text, gid integer); mysql> CREATE CLUSTER posts; Query OK, 0 rows affected (0.24 sec) mysql> ALTER CLUSTER posts ADD testrt; Query OK, 0 rows affected (0.07 sec) MySQL [(none)]> exit Bye
-
Join to the the cluster on the 2nd instance
$ docker-compose exec manticore-2 mysql mysql> JOIN CLUSTER posts AT 'manticore-1:9312'; mysql> INSERT INTO posts:testrt(title,content,gid) VALUES('hello','world',1); Query OK, 1 row affected (0.00 sec) MySQL [(none)]> exit Bye
-
If you now go back to the first instance you'll see the new record:
$ docker-compose exec manticore-1 mysql MySQL [(none)]> select * from testrt; +---------------------+------+-------+---------+ | id | gid | title | content | +---------------------+------+-------+---------+ | 3891565839006040065 | 1 | hello | world | +---------------------+------+-------+---------+ 1 row in set (0.00 sec) MySQL [(none)]> exit Bye
It's recommended to overwrite the default ulimits of docker for the Manticore instance:
--ulimit nofile=65536:65536
For best performance, table components can be "mlocked" into memory. When Manticore is run under Docker, the instance requires additional privileges to allow memory locking. The following options must be added when running the instance:
--cap-add=IPC_LOCK --ulimit memlock=-1:-1
If you want to run Manticore with a custom configuration that includes table definitions, you will need to mount the configuration to the instance:
docker run -e EXTRA=1 --name manticore -v $(pwd)/manticore.conf:/etc/manticoresearch/manticore.conf -v $(pwd)/data/:/var/lib/manticore -p 127.0.0.1:9306:9306 -d manticoresearch/manticore
Take into account that Manticore search inside the container is run under user manticore
. Performing operations with table files (like creating or rotating plain tables) should be also done under manticore
. Otherwise the files will be created under root
and the search daemon won't have rights to open them. For example here is how you can rotate all tables:
docker exec -it manticore gosu manticore indexer --all --rotate
You can also set individual searchd
and common
configuration settings using Docker environment variables.
The settings must be prefixed with their section name, example for in case of mysql_version_string
the variable must be named searchd_mysql_version_string
:
docker run -e EXTRA=1 --name manticore -p 127.0.0.1:9306:9306 -e searchd_mysql_version_string='5.5.0' -d manticoresearch/manticore
In case of the listen
directive, new listening interfaces using the Docker variable searchd_listen
in addition to the default ones. Multiple interfaces can be declared, separated by a semi-colon ("|"). To listen only on a network address, the $ip
(retrieved internally from hostname -i
) can be used as address alias.
For example -e searchd_listen='9316:http|9307:mysql|$ip:5443:mysql_vip'
will add an additional SQL interface on port 9307, an SQL VIP listener on port 5443 running only on the instance's IP, and an HTTP listener on port 9316, in addition to the defaults on 9306 and 9308, respectively.
$ docker run -e EXTRA=1 --rm -p 1188:9307 -e searchd_mysql_version_string='5.5.0' -e searchd_listen='9316:http|9307:mysql|$ip:5443:mysql_vip' manticore
[Mon Aug 17 07:31:58.719 2020] [1] using config file '/etc/manticoresearch/manticore.conf' (9130 chars)...
listening on all interfaces for http, port=9316
listening on all interfaces for mysql, port=9307
listening on 172.17.0.17:5443 for VIP mysql
listening on all interfaces for mysql, port=9306
listening on UNIX socket /var/run/mysqld/mysqld.sock
listening on 172.17.0.17:9312 for sphinx
listening on all interfaces for http, port=9308
prereading 0 indexes
prereaded 0 indexes in 0.000 sec
accepting connections
To start Manticore with custom startup flags, specify them as arguments when using docker run. Ensure you do not include the searchd
command and include the --nodetach
flag. Here's an example:
docker run -e EXTRA=1 --name manticore --rm manticoresearch/manticore:latest --replay-flags=ignore-trx-errors --nodetach
By default, the main Manticore process searchd
is running under user manticore
inside the container, but the script which runs on starting the container is run under your default docker user which in most cases is root
. If that's not what you want you can use docker ... --user manticore
or user: manticore
in docker compose yaml to make everything run under manticore
. Read below about possible volume permissions issue you can get and how to solve it.
To build plain tables specified in your custom configuration file, you can use the CREATE_PLAIN_TABLES=1
environment variable. It will execute indexer --all
before Manticore starts. This is useful if you don't use volumes, and your tables are easy to recreate.
docker run -e CREATE_PLAIN_TABLES=1 --name manticore -v $(pwd)/manticore.conf:/etc/manticoresearch/manticore.conf -p 9306:9306 -p 9308:9308 -d manticoresearch/manticore
In case you are running Manticore Search docker under non-root (using docker ... --user manticore
or user: manticore
in docker compose yaml), you can face a permissions issue, for example:
FATAL: directory /var/lib/manticore write error: failed to open /var/lib/manticore/tmp: Permission denied
or in case you are using -e EXTRA=1
:
mkdir: cannot create directory ‘/var/lib/manticore/.mcl/’: Permission denied
This can happen because the user which is used to run processes inside the container may have no permissions to modify the directory you have mounted to the container. To fix it you can chown
or chmod
the mounted directory. If you run the container under user manticore
you need to do:
chown -R 999:999 data
since user manticore
has ID 999 inside the container.
On Windows, if you want Manticore to start at boot, you can install it as a Windows Service. You can follow the instructions in the Manticore as Windows Service guide to install Manticore as a service.
Once Manticore is installed as a service, you can start and stop it from the Control Panel or from the command line using the sc.exe command.
sc.exe start Manticore
sc.exe stop Manticore
Alternatively, if you don't install Manticore as a Windows service, you can start it from the command line by running the following command:
.\bin\searchd -c manticore.conf
This command assumes that you have the Manticore's binary and the configuration file in the current directory.