MySQL protocol

Manticore Search implements an SQL interface using the MySQL protocol, allowing any MySQL library or connector and many MySQL clients to be used to connect to Manticore Search and work with it as if it were a MySQL server, not Manticore.

However, the SQL dialect is different and implements only a subset of the SQL commands or functions available in MySQL. Additionally, there are clauses and functions that are specific to Manticore Search, such as the MATCH() clause for full-text search.

Manticore Search does not support server-side prepared statements, but client-side prepared statements can be used. It is important to note that Manticore implements the multi-value (MVA) data type, which has no equivalent in MySQL or libraries implementing prepared statements. In these cases, the MVA values must be crafted in the raw query.

Some MySQL clients/connectors require values for user/password and/or database name. Since Manticore Search does not have the concept of databases and there is no user access control implemented, these values can be set arbitrarily as Manticore will simply ignore them.

Configuration

The default port for the SQL interface is 9306 and it's enabled by default.

You can configure the MySQL port in the searchd section of the configuration file using the listen directive like this:

searchd {
...
   listen = 127.0.0.1:9306:mysql
...
}

Keep in mind that Manticore doesn't have user authentication, so make sure that the MySQL port is not accessible to anyone outside of your network.

VIP connection

A separate MySQL port can be used for performing "VIP" connections. When connecting to this port, the thread pool is bypassed, and a new dedicated thread is always created. This is useful in cases of severe overload, where the server would either stall or prevent a connection through the regular port.

searchd {
...
   listen = 127.0.0.1:9306:mysql
   listen = 127.0.0.1:9307:mysql_vip
...
}

Connecting via standard MySQL client

The easiest way to connect to Manticore is by using a standard MySQL client:

mysql -P9306 -h0

Secured MySQL connection

The MySQL protocol supports SSL encryption. Secure connections can be made on the same mysql listening port.

Compressed MySQL connection

Compression can be used with MySQL connections and is available to clients by default. The client just needs to specify that the connection should use compression.

An example using the MySQL client:

mysql -P9306 -h0 -C

Compression can be used in both secured and non-secured connections.

Notes on MySQL connectors

The official MySQL connectors can be used to connect to Manticore Search, however they might require certain settings passed in the DSN string as the connector can try running certain SQL commands not implemented yet in Manticore.

JDBC Connector 6.x and above require Manticore Search 2.8.2 or greater and the DSN string should contain the following options:

jdbc:mysql://IP:PORT/DB/?characterEncoding=utf8&maxAllowedPacket=512000&serverTimezone=XXX

By default Manticore Search will report it's own version to the connector, however this may cause some troubles. To overcome that mysql_version_string directive in searchd section of the configuration should be set to a version lower than 5.1.1:

searchd {
...
   mysql_version_string = 5.0.37
...
}

.NET MySQL connector uses connection pools by default. To correctly get the statistics of SHOW META, queries along with SHOW META command should be sent as a single multistatement (SELECT ...;SHOW META). If pooling is enabled option Allow Batch=True is required to be added to the connection string to allow multistatements:

Server=127.0.0.1;Port=9306;Database=somevalue;Uid=somevalue;Pwd=;Allow Batch=True;

Notes on ODBC connectivity

Manticore can be accessed using ODBC. It's recommended to set charset=UTF8 in the ODBC string. Some ODBC drivers will not like the reported version by the Manticore server as they will see it as a very old MySQL server. This can be overridden with mysql_version_string option.

Comment syntax

Manticore SQL over MySQL supports C-style comment syntax. Everything from an opening /* sequence to a closing */ sequence is ignored. Comments can span multiple lines, can not nest, and should not get logged. MySQL specific /*! ... */ comments are also currently ignored. (As the comments support was rather added for better compatibility with mysqldump produced dumps, rather than improving general query interoperability between Manticore and MySQL.)

SELECT /*! SQL_CALC_FOUND_ROWS */ col1 FROM table1 WHERE ...

HTTP

You can connect to Manticore Search through HTTP/HTTPS.

Configuration

By default, Manticore listens for HTTP, HTTPS, and binary requests on ports 9308 and 9312.

In the "searchd" section of your configuration file, you can define the HTTP port using the listen directive as follows:

Both lines are valid and have the same meaning (except for the port number). They both define listeners that will serve all API/HTTP/HTTPS protocols. There are no special requirements, and any HTTP client can be used to connect to Manticore.

‹›
  • HTTP
HTTP
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9312:http
...
}

All HTTP endpoints return application/json content type. For the most part, endpoints use JSON payloads for requests. However, there are some exceptions that use NDJSON or simple URL-encoded payloads.

Currently, there is no user authentication. Therefore, make sure that the HTTP interface is not accessible to anyone outside your network. As Manticore functions like any other web server, you can use a reverse proxy, such as Nginx, to implement HTTP authentication or caching.

The HTTP protocol also supports SSL encryption: If you specify :https instead of :http only secured connections will be accepted. Otherwise in case of no valid key/certificate provided, but the client trying to connect via https - the connection will be dropped. If you make not HTTPS, but an HTTP request to 9443 it will respond with HTTP code 400.

‹›
  • HTTPS
HTTPS
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9443:https
...
}

VIP Connection

Separate HTTP interface can be used for 'VIP' connections. In this case, the connection bypasses a thread pool and always creates a new dedicated thread. This is useful for managing Manticore Search during periods of severe overload when the server might stall or not allow regular port connections.

For more information on the listen directive, see this section.

‹›
  • VIP
VIP
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9318:_vip
...
}

SQL over HTTP

Endpoints /sql and /cli allow running SQL queries via HTTP.

  • /sql endpoint accepts only SELECT statements and returns the response in HTTP JSON format. The query parameter should be URL-encoded.
  • The /sql?mode=raw endpoint accepts any SQL query and returns the response in raw format, similar to what you would receive via mysql. The query parameter should also be URL-encoded.
  • The /cli endpoint accepts any SQL query and returns the response in raw format, similar to what you would receive via mysql. Unlike the /sql and /sql?mode=raw endpoints, the query parameter should not be URL-encoded. This endpoint is intended for manual actions using a browser or command line HTTP clients such as curl. It is not recommended to use the /cli endpoint in scripts.

/sql

/sql accepts an SQL SELECT query via HTTP JSON interface.

Query payload must be URL encoded, otherwise query statements with = (filtering or setting options) will result in an error.

It returns a JSON response which contains hits information and execution time. The response has the same format as json/search endpoint. Note, that /sql endpoint supports only single search requests. If you are looking for processing a multi-query see below.

‹›
  • HTTP
HTTP
📋
POST /sql -d "query=select%20id%2Csubject%2Cauthor_id%20%20from%20forum%20where%20match%28%27%40subject%20php%20manticore%27%29%20group%20by%20author_id%20order%20by%20id%20desc%20limit%200%2C5"
‹›
Response
{
  "took": 0,
  "timed_out": false,
  "hits": {
    "total": 2,
    "total_relation": "eq",
    "hits": [
      {
        "_id": "2",
        "_score": 2356,
        "_source": {
          "subject": "php manticore",
          "author_id": 12
        }
      },
      {
        "_id": "1",
        "_score": 2356,
        "_source": {
          "subject": "php manticore",
          "author_id": 11
        }
      }
    ]
  }
}

/sql?mode=raw

/sql endpoint also has a special mode "raw", which allows to send any valid sphinxql queries including multi-queries. The returned value is a json array of one or more result sets.

‹›
  • HTTP
HTTP
📋
POST /sql?mode=raw -d "query=desc%20test"
‹›
Response
[
  {
    "columns": [
      {
        "Field": {
          "type": "string"
        }
      },
      {
        "Type": {
          "type": "string"
        }
      },
      {
        "Properties": {
          "type": "string"
        }
      }
    ],
    "data": [
      {
        "Field": "id",
        "Type": "bigint",
        "Properties": ""
      },
      {
        "Field": "title",
        "Type": "text",
        "Properties": "indexed"
      },
      {
        "Field": "gid",
        "Type": "uint",
        "Properties": ""
      },
      {
        "Field": "title",
        "Type": "string",
        "Properties": ""
      },
      {
        "Field": "j",
        "Type": "json",
        "Properties": ""
      },
      {
        "Field": "new1",
        "Type": "uint",
        "Properties": ""
      }
    ],
    "total": 6,
    "error": "",
    "warning": ""
  }
]

/cli

While the /sql endpoint is useful to control Manticore programmatically from your application, there's also endpoint /cli which makes it easier to maintain a Manticore instance via curl or your browser manually. It accepts POST and GET HTTP methods. Everything after /cli? is taken by Manticore as is, even if you don't escape it manually via curl or let the browser encode it automatically. The + sign is not decoded to a space as well, eliminating the necessity of encoding it. The response format is tabular, similar to the one returned by MySQL console.

‹›
  • HTTP
  • Browser
📋
POST /cli -d "desc test"
‹›
Response
+-------+--------+----------------+
| Field | Type   | Properties     |
+-------+--------+----------------+
| id    | bigint |                |
| body  | text   | indexed stored |
| title | string |                |
+-------+--------+----------------+
3 rows in set (0.001 sec)

/cli_json

The /cli_json endpoint provides the same functionality as /cli , but returns the response in JSON format.

‹›
  • HTTP
HTTP
📋
POST /cli_json -d "desc test"
‹›
Response
[{
"columns":[{"Field":{"type":"string"}},{"Type":{"type":"string"}},{"Properties":{"type":"string"}}],
"data":[
{"Field":"id","Type":"bigint","Properties":""},
{"Field":"body","Type":"text","Properties":"indexed stored"},
{"Field":"title","Type":"string","Properties":""}
],
"total":3,
"error":"",
"warning":""
}]

Keep-alive

HTTP keep-alive is also supported, which makes working via the HTTP JSON interface stateful as long as the client supports keep-alive too. For example, using the new /cli endpoint you can call SHOW META after SELECT and it will work the same way it works via mysql.

HTTP

You can connect to Manticore Search through HTTP/HTTPS.

Configuration

By default, Manticore listens for HTTP, HTTPS, and binary requests on ports 9308 and 9312.

In the "searchd" section of your configuration file, you can define the HTTP port using the listen directive as follows:

Both lines are valid and have the same meaning (except for the port number). They both define listeners that will serve all API/HTTP/HTTPS protocols. There are no special requirements, and any HTTP client can be used to connect to Manticore.

‹›
  • HTTP
HTTP
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9312:http
...
}

All HTTP endpoints return application/json content type. For the most part, endpoints use JSON payloads for requests. However, there are some exceptions that use NDJSON or simple URL-encoded payloads.

Currently, there is no user authentication. Therefore, make sure that the HTTP interface is not accessible to anyone outside your network. As Manticore functions like any other web server, you can use a reverse proxy, such as Nginx, to implement HTTP authentication or caching.

The HTTP protocol also supports SSL encryption: If you specify :https instead of :http only secured connections will be accepted. Otherwise in case of no valid key/certificate provided, but the client trying to connect via https - the connection will be dropped. If you make not HTTPS, but an HTTP request to 9443 it will respond with HTTP code 400.

‹›
  • HTTPS
HTTPS
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9443:https
...
}

VIP Connection

Separate HTTP interface can be used for 'VIP' connections. In this case, the connection bypasses a thread pool and always creates a new dedicated thread. This is useful for managing Manticore Search during periods of severe overload when the server might stall or not allow regular port connections.

For more information on the listen directive, see this section.

‹›
  • VIP
VIP
📋
searchd {
...
   listen = 127.0.0.1:9308
   listen = 127.0.0.1:9318:_vip
...
}

SQL over HTTP

Endpoints /sql and /cli allow running SQL queries via HTTP.

  • /sql endpoint accepts only SELECT statements and returns the response in HTTP JSON format. The query parameter should be URL-encoded.
  • The /sql?mode=raw endpoint accepts any SQL query and returns the response in raw format, similar to what you would receive via mysql. The query parameter should also be URL-encoded.
  • The /cli endpoint accepts any SQL query and returns the response in raw format, similar to what you would receive via mysql. Unlike the /sql and /sql?mode=raw endpoints, the query parameter should not be URL-encoded. This endpoint is intended for manual actions using a browser or command line HTTP clients such as curl. It is not recommended to use the /cli endpoint in scripts.

/sql

/sql accepts an SQL SELECT query via HTTP JSON interface.

Query payload must be URL encoded, otherwise query statements with = (filtering or setting options) will result in an error.

It returns a JSON response which contains hits information and execution time. The response has the same format as json/search endpoint. Note, that /sql endpoint supports only single search requests. If you are looking for processing a multi-query see below.

‹›
  • HTTP
HTTP
📋
POST /sql -d "query=select%20id%2Csubject%2Cauthor_id%20%20from%20forum%20where%20match%28%27%40subject%20php%20manticore%27%29%20group%20by%20author_id%20order%20by%20id%20desc%20limit%200%2C5"
‹›
Response
{
  "took": 0,
  "timed_out": false,
  "hits": {
    "total": 2,
    "total_relation": "eq",
    "hits": [
      {
        "_id": "2",
        "_score": 2356,
        "_source": {
          "subject": "php manticore",
          "author_id": 12
        }
      },
      {
        "_id": "1",
        "_score": 2356,
        "_source": {
          "subject": "php manticore",
          "author_id": 11
        }
      }
    ]
  }
}

/sql?mode=raw

/sql endpoint also has a special mode "raw", which allows to send any valid sphinxql queries including multi-queries. The returned value is a json array of one or more result sets.

‹›
  • HTTP
HTTP
📋
POST /sql?mode=raw -d "query=desc%20test"
‹›
Response
[
  {
    "columns": [
      {
        "Field": {
          "type": "string"
        }
      },
      {
        "Type": {
          "type": "string"
        }
      },
      {
        "Properties": {
          "type": "string"
        }
      }
    ],
    "data": [
      {
        "Field": "id",
        "Type": "bigint",
        "Properties": ""
      },
      {
        "Field": "title",
        "Type": "text",
        "Properties": "indexed"
      },
      {
        "Field": "gid",
        "Type": "uint",
        "Properties": ""
      },
      {
        "Field": "title",
        "Type": "string",
        "Properties": ""
      },
      {
        "Field": "j",
        "Type": "json",
        "Properties": ""
      },
      {
        "Field": "new1",
        "Type": "uint",
        "Properties": ""
      }
    ],
    "total": 6,
    "error": "",
    "warning": ""
  }
]

/cli

While the /sql endpoint is useful to control Manticore programmatically from your application, there's also endpoint /cli which makes it easier to maintain a Manticore instance via curl or your browser manually. It accepts POST and GET HTTP methods. Everything after /cli? is taken by Manticore as is, even if you don't escape it manually via curl or let the browser encode it automatically. The + sign is not decoded to a space as well, eliminating the necessity of encoding it. The response format is tabular, similar to the one returned by MySQL console.

‹›
  • HTTP
  • Browser
📋
POST /cli -d "desc test"
‹›
Response
+-------+--------+----------------+
| Field | Type   | Properties     |
+-------+--------+----------------+
| id    | bigint |                |
| body  | text   | indexed stored |
| title | string |                |
+-------+--------+----------------+
3 rows in set (0.001 sec)

/cli_json

The /cli_json endpoint provides the same functionality as /cli , but returns the response in JSON format.

‹›
  • HTTP
HTTP
📋
POST /cli_json -d "desc test"
‹›
Response
[{
"columns":[{"Field":{"type":"string"}},{"Type":{"type":"string"}},{"Properties":{"type":"string"}}],
"data":[
{"Field":"id","Type":"bigint","Properties":""},
{"Field":"body","Type":"text","Properties":"indexed stored"},
{"Field":"title","Type":"string","Properties":""}
],
"total":3,
"error":"",
"warning":""
}]

Keep-alive

HTTP keep-alive is also supported, which makes working via the HTTP JSON interface stateful as long as the client supports keep-alive too. For example, using the new /cli endpoint you can call SHOW META after SELECT and it will work the same way it works via mysql.